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Chapter 1: The Basics 


Overview 


This chapter is a guide to the Unix commands and utilities that you must know to get anything out of this 
book. This is preliminary material, and you may already know a substantial amount. Even if you think 
you're up to speed, though, take a few seconds to flip through the sections just to make sure. 


You may be asking yourself, "I thought this was a book about Linux, not Unix?" Don't worry; this chapter is 
just getting you ready for working with the Linux system. Linux is a Unix flavor at heart. You will see the 
word "Unix" in this chapter more than "Linux" because you can take the information straight over to 
Solaris, BSD, and other Unix-flavored systems. Special care has been taken to avoid too many Linux- 
specific user interface extensions, not only so you will have a better background with other operating 
systems, but also because these extensions tend to be extremely unstable. You will be able to adapt to 
new Linux releases much more quickly if you know the core that does not change. 


Although the material here may seem sparse, you need to convince yourself that Unix is not hard. Yes, 
most reports from the field are to the contrary, but in the end, Unix is nothing but a bunch of files and a few 
commands for manipulating those files. If you are familiar with other operating systems, you shouldn't 
have a problem. However, if you find the information in this preliminary chapter somewhat lacking, there 
are books with much more detail for beginners, such as UNIX for the Impatient [Abrahams] and Learning 
the UNIX Operating System [Peek]. 


1.1 About /bin/sh 


The shell is one of the most important parts of a Unix system. A shell is a program that runs commands. 
For example, one of the shell's duties is to run the commands that users type. Shells also serve as small 
programming environments. Unix programmers often break common tasks into little components and rely 
on the shell to manage tasks and to piece things together. 


Many important parts of the system are actually shell scripts — text files that contain nothing but shell 
commands. If you have worked with MS-DOS, shell scripts may seem similar to . BAT files, but the shell is 
far more powerful. Chapter 7 is a small guide to shell scripts, and you can peruse it anytime after finishing 
this chapter. 


As you progress through this book, you will learn how to manipulate commands with the shell. One of the 
best things about the shell is that if you make a mistake, you can look at what you typed, see what went 
wrong, and then try again quickly. Do not be afraid to try new things. The only way to learn the shell is to 
use it. 


There are many different Unix shells, but all derive many of their features from the Bourne shell, or 
/bin/sh. Every Unix system needs the Bourne shell to function correctly, as you will see throughout this 
book. 


Linux uses an enhanced version of this shell, called bash, or the "Bourne-again" shell. bash is the default 
shell for most Linux distributions, and /bin/sh is normally some sort of link to bash ona Linux system. 
You should use the bash shell when running the examples in this book. 


You may not have bash if you're using this chapter as a guide for a Unix account at an organization where 
you are not the systems administrator. You can change your shell with chsh or ask your systems 
administrator for help. 


1.2Using the Shell 


When you installed Linux, you set up a root (Superuser) password, and hopefully you also made at least 
one more regular user for yourself. For this chapter, you should log in as the regular user. 


The first thing you need to do after logging in is to bring up a shell window. After starting a shell, its window 
contains a prompt at the top that usually ends with a dollar sign ($). On Red Hat Linux, the prompt looks 
like [name@host path]S. If you know Windows, you will find that the shell window looks vaguely similar 
to the command prompt. 


Note You may hear the shell window referred to as the terminal window. The terminal window is the 
part of the user interface that accepts keystrokes from the windowing system and draws the 
characters on the screen. You can think of the terminal as a middleman between the shell and 
the user. 


Now type the following command and press ENTER: 


cat /etc/passwd 


The command prints a couple of lines that start with usernames, and then you get your shell prompt back. 
If you can't read the text very well, manipulate the font settings to your liking. You're going to be spending 
a good part of your time with this book in the shell, so you should be comfortable with it. 


Thecat command is a great one to start with because it's one of the easiest Unix commands. Its syntax is 
as follows: 


catfilel file2 ... 


When you run this command, cat prints (and concatenates) the contents of filel,file2, and any 
additional files you desire (denoted by . . .) and exits. 


If you do not specify any input files, cat reads from the standard input, which in this case is the keyboard. 
To see this at work, type cat and press ENTER. Unlike the earlier example, you do not get your shell 
prompt back because cat is still running. Now type some stuff (it doesn't matter what). After you press 
ENTER at the end of each line, cat repeats the line you typed. When you're sick of this, press CONTROL- 
D on a line by itself to terminate the cat command and return to the shell prompt. 


Note Do not confuse CONTROL-D and CONTROL-C. CONTROL-D on aline by itself stops the 
current standard input entry (and often terminates a program). CONT ROL-C terminates a 
program regardless of its input or output. 


You have now experienced standard input (stdin) and output (stdout), two important components of the 
Unix shell environment. Standard input is a program's default input source. When you ran cat without any 
arguments, it didn't give up because there were no arguments; instead, cat switched to standard input 
instead of going after some files. Here, standard input was the stuff that came from your keyboard. 
Standard output, on the other hand, is where a program's output goes by default. In this case, that location 
is the terminal window running the shell. The best feature of standard input and output is that you can 
easily send them to places other than their defaults, as described in Section 1.14. 


1.3Basic Commands 


It's time to learn some other Unix commands. Most of the following programs take multiple arguments, and 
some have so many options and formats that an unabridged listing would be pointless. This is a simplified 
list; you don't need to know all of the details just yet. 


1.3.1ls 


Thels command lists the contents of a directory. The default is the current directory. Use 1s -1 fora 
detailed (long) listing and 1s -F to display file type information (for more on file types and the permissions 
in the left column, see Section 1.17). Here is asample long listing: 


total 3616 
=rw-r--r-- 
-rw-r--r-- 
-rw-r--r-- 
=rwWw-r--r-- 


drwxr-xr-x 
—1rwxr-xr—x 
“fy 1S 
=W- r- r= 
drwxr-xr-x 
drwxr-xr-x 


1.3.2cp 


WOARRFPERENFRFRE EE 


jJuser 
jJuser 
jJuser 
jJuser 
JUser 
jJuser 
jJuser 
jJuser 
jJuser 
jJuser 


users 
users 
users 
users 
users 
users 
users 
users 
users 
users 


3804 
4165 


LoLa de 


6295 
4096 
7 OG 
Lau? 

56 
4096 
4096 


Apr 
May 
Oct 
May 
Jul 
Feb 
Oct 
Oct 
Feb 
May 


30 
26 
26 
30 
17 


20 


20 
2 


2000 
Loo? 
Z000 
1999 
20g 
2001 
1999 
1999 
Lood 
2000 


abusive.c 
battery.zip 
beav_1.40-13.tar.gz 
country.c 

GSS 

dhry 

Chey ac 

doit 

dw 

hough—sturir 


In the first form shown below, cp copies the contents of file1 to file2. In the second form, it copies all 
files to the dir directory: 


cpfilel flle 
fileN dir 


cpfilel 


1.3.3mv 


In the first form below, mv renames filel to file2. In the second form, it moves all files to the dir 


directory: 


mvfileil file2 
fileN dir 


mvfiliel 


1.3.4touch 


Thet ouch command creates a file. If the file already exists, touch does not change it, but it does update 
the timestamp you see with the long listing that you get with the 1s -1 command. 


touchfile 


1.3.5rm 


To delete (remove) a file, use rm. After you remove a file, it's gone. Do not expect to be able to "undelete" 


anything. 


rmfile 


1.3.6echo 


Theecho command prints its arguments to the standard output: 


echo Hello there. 


The echo command is very useful for finding expansions of shell wildcards and variables that you will 
encounter later in this chapter. 


1.4Using Directory Commands 


Unix has a directory hierarchy that starts at /, sometimes called the root. The directory separator is the 
slash (/),notthe backslash (\). There are several standard subdirectories in the root directory, such as 
/usr (you'll learn all about them in Section 2.1). 


A directory specification is called a path, and one that starts at the root (such as /usr/1ib) isa fullor 
absolute path. Similarly, a filename with a full path in front (such as /usr/lib/libc.a) isa full 
pathname. 

The path component identified by two dots (. .) specifies the parent of your shell's current directory, and 
one dot (.) specifies the current directory. For example, if the current working directory of your shell is 
/usr/lib, the path ../binrefersto /usr/bin. A path beginning with .. or. is called a relative 
pathname. 


The following sections describe the essential directory commands. 


1.4.1¢cd 


Thecd command changes the shell's current working directory to dir: 


cdair 


If you omit dir, the shell returns to your home directory. 


1.4.2mkdir 


Themkdir command creates anew directory, dir: 


mkdirdir 


1.4.3rmdir 


Thermdir command removes the directory dir: 


rmdirdir 


Ifair isn't empty, this command fails. However, if you're impatient, you probably don't want to laboriously 
delete all the files and subdirectories inside dir first. You can use rm -rf dir to delete a directory and 
its contents, but be careful. This is one of the few commands that can do serious damage, especially if you 
run it as the superuser. The -r option specifies recursive delete, and -f forces the delete operation. Don't 
use the -rf flags with wildcards such as a star (*). Above all, always double-check your command. 


1.4.4Shell Wildcards 


The shell is capable of matching simple patterns with files in the current working directory. The simplest of 
these is the star character (*), which means match any number of arbitrary characters. For example, the 
following command prints a list of files in the current directory: 


echo * 


After matching files to wildcards, the shell substitutes the filenames for the wildcard in the command line 
and then runs the revised command line. Here are some more wildcard examples: at * matches all files 
starting with at;*at matches files that end with at; and *at* matches any files that contains at. If no 
files match a wildcard, the shell does no substitution, and the command runs with literal characters such as 
x (for example, try a command such as echo *dfkdsafh). 


If you're used to MS-DOS, you might instinctively type *. * as a wildcard to match all files. Break this habit 
now. In Linux and other versions of Unix, you must use * to match all files. In the Unix shell, * . * matches 
only files and directories that contain the dot (.) character in their names. Unix filenames do not need 
extensions and often do not carry them. 


Another shell wildcard character is the question mark (7), instructing the shell to match exactly one 
arbitrary character. For example, b?at matches boat andbrat. 


If you do not want the shell to expand a wildcard in a command, enclose the wildcard in single quotes (''). 
For example, the command echo '*' prints a star. You will find this handy in a few of the commands 
described in the next section, such as grep and find. Quoting is a somewhat tricky matter, so don't get 
too involved with it just yet — you'll learn more much later, in Section 7.2. 


This isn't the end to a modern shell's pattern-matching capabilities, but * and ? are what you need to know. 


1.5Intermediate Commands 


The following sections describe the most essential intermediate Unix commands beyond the basics that 
you saw earlier in this chapter. There are many more commands where these come from; check out 
Appendix A for a large list of commands on your system. 


1.5.1grep 


grep prints the lines from a file or input stream that match an expression. For example, if you want to print 
the lines in the /etc/passwd file that contain the text root, use this command: 


grep root /etc/passwd 


Thegrep command is extraordinarily handy when operating on multiple files at once, because it prints the 
filename in addition to the matching line when in this multiple-file mode. For example, if you want to check 
on every file in /etc that contains root, you could use this command: 


grep root /etc/* 


Two of the most important grep options are -i (for case-insensitive matches) and -v (which inverts the 
search; that is, it prints all lines that don't match). There is also a more powerful variant called egrep. 


grep understands patterns known as regular expressions that are grounded in computer science theory 
and are ubiquitous in Unix utilities. Regular expressions are more powerful than wildcard-style patterns, 
and they have a different syntax. The two most important things to remember about regular expressions 
are these: 


=» .* to match any number of characters (like the * in wildcards) 
a . to match one arbitrary character 


The grep(1) manual page contains a detailed description of regular expressions, but it can be a little 
difficult to read. To learn more, you can try Mastering Regular Expressions [Friedl], or look at the regular 
expressions chapter of Programming Perl [Wall]. If you like math and are interested in where these things 
come from, look up /ntroduction to Automata Theory, Languages, and Computation [Hopcroft]. 


1.5.2more and less 


When a command's output is long, it can scroll off the top of the screen, and it's annoying to use a 
scrollbar to view such output because you have to move your hands around. You sometimes may also 
want to look at a large text file without starting a text editor. Two standard commands for text navigation 
aremore and less. 


To page through a big file like /usr/dict/words, use acommand such aS more /usr/dict/words. 
When running more, you will see the contents of the file, one screenful at a time. You can press the space 
bar to go forward in the file and the b key to skip back one screenful. To quit more, type q. 


Theless command performs the same function as more, but it is far more powerful and widely used. Use 
less --help to geta good summary of its operations. 


As you will learn in Section 1.14, you can send the output of nearly any program directly to another 
program's input, enabling operations such as this (try it to see what it does): 


grep ie /usr/dict/words | less 


1.5.3pwd 


This program's name stands for "print working directory," and the command outputs the current working 
directory. That's all that pwd does, but it's useful. Some Linux distributions set up accounts with the current 
working directory in the prompt, but you may wish to change that because the current working directory 


takes up a lot of space on a line. 


1.5.4diff 


To see the differences between two text files, use diff: 


difffilei file2 


There are several options that can control the format of the output, such as —c, but the default output 
format is often the most comprehensible (for human beings, that is). 


1.5.5file 


If you see a file and are unsure of its format, try using file to see if the system can guess, based on a 
large set of rules: 


filefile 


You may be surprised to see how much this innocent-looking command can do. 


1.5.6find 


It's frustrating when you know that a certain file is in a directory tree somewhere, and you just don't know 
where. Run findtofind filein dir: 


finddir -name file -print 


Like most programs in this section, find is capable of some fancy stuff. However, don't try options such 
as—exec before you know the form shown here by heart, and you know why you need the -name and — 
print options. The find command accepts wildcard characters, such as *, but you must enclose them in 
single quotes ('*') to protect the wildcard characters from the shell's own wildcard features (recall from 
Section 1.4.4 that the shell expands wildcards before running commands). 


1.5.7head and tail 


To quickly view a portion of a file or stream, use the head and tail commands. For example, head 
/etc/inittab shows the first ten lines of this system configuration file, and tail /etc/inittab 
shows the last ten lines. You can change the number of lines to print by using the -n option, where n is the 
number of lines you want to see. If you want to print lines starting at line n, use tail +n. 


1.5.8sort 


Thesort command quickly puts the lines of a text file in alphanumeric order. If the file's lines start with 
numbers, and you want to sort in numeric order, use the -n option. The -r option reverses the order of the 
sort. 


1.6Changing Your Password and Shell 


On large commercial sites, it is important to change your password when you first log in. At home, you 
may not even have a password, but you should. Use the passwd command to change your password. It 


asks for your old password and then prompts you for your new password twice. 


As any other manual will tell you, choose a password that does not include any real words in any language. 
Don't try to combine words, either. One of the easiest ways to get a good password is to pick a sentence, 
produce an acronym from it, and then modify the acronym with a number or some punctuation. All you 
have to do is remember the sentence. For example, Ayht8irt is a modification of the previous sentence. 


1.6.1chsh 


You can change your shell with the chsh command, but keep in mind that this book assumes that you're 
runningbash. 


1.7 Dot Files 


Change to your home directory and take a look around with 1s, and then run 1s -a. At first, you will not 
see the configuration files, which are also called dotfiles. Dot files are nothing more than files and 
directories whose names begin with a dot (.). Common dot files are .bashrc and .login. There are 
some dot directories, too, such as .mozilla. 


There is nothing special about dot files or directories. Some programs just don't list them by default, so 
that you don't see a complete mess when listing the contents of your home directory. For example, without 
the-a option, 1s doesn't list dot files. In addition, shell wildcards don't match dot files unless you explicitly 


use a pattern suchas .*. 


Note You can still run into problems with wildcards because .* matches . and .. (the current and 


parent directories). Depending on what you're doing, you may wish to use a pattern such as 
.[^.]* or .??* to get all dot files exceptthe current and parent directories. This isn't a perfect 


solution, but it usually works. 


1.8Environment and Shell Variables 


The shell can store temporary variables, called shell variables, that store the values of text strings. Shell 
variables are very useful for keeping track of state in scripts, and some shell variables control the way the 
shell behaves (for example, the PS1 variable controls the prompt). To assign a value to a shell variable, 
use the equal sign (=): 


STUFF=blah 


The preceding example sets the value of the variable named STUFF to blah. To access this variable, use 
SSTUFF (for example, try running echo $STUFF). 


Anenvironment variable is like a shell variable, but is not specific to the shell. All programs on Unix 
systems have environment variable storage. The difference that you will notice is that the operating 
system passes all of your shell's environment variables to programs that the shell runs, whereas shell 
variables are not accessible by the commands that you run. Some programs use environment variables for 
configuration and options. For example, you can put your favorite less command-line options in the LESS 
environment variable, and less will use these when you run it. Many manual pages contain a section 
marked ENVIRONMENT that describes these variables. 


You can assign a new environment variable just as you would a shell variable, except that after creating 
the variable, you must run the export operation to transfer the variable to the shell's environment 
variable storage. The following sequence of commands assigns a value to STUFF and changes it into an 
environment variable: 


STUFF=blah 
export STUFF 


1.9The Command Path 


PATH is a special environment variable containing the command path (or path for short). A command path 
is a list of system directories that the shell looks in when trying to locate a command. For example, if you 
try to run the 1s command, the shell searches the directories listed in PATH for the 1s program. If 
programs with the same name appear in several directories in the path, the shell runs the first program 
that matches. 


If you run echo $PATH, you'll see that the path components are separated by colons (:). Here's a simple 
example: 


Jusr/ local/ bin: /usr/XlIRG/bin:/usr/ bin: / bin 


To make the shell look in more places for programs, you can change the PATH environment variable. You 
can add a directory dir to the beginning of the path so that the shell looks in dir before looking in any of 
the other PATH directories with this command: 


PATH=01 7r:SPATH 


As an alternative, you can append a directory name to the end of the PATH variable, causing the shell to 
look in dir last: 


PATH=SPATH: dir 


Note Exercise caution when modifying the path, because you can accidentally wipe out your entire 
path if you mistype $PATH. Don't panic if this happens, because it isn't permanent (for a lasting 
effect, you need to mistype it when editing a certain configuration file, and even then it isn't 
difficult to rectify). One of the easiest methods to get back to normal is to exit the terminal 
window that you're using and start another. 


1.10Special Characters 


If you Know people who are into Linux, and you have some inexplicable desire to discuss it with them, you 
should know a few names for some of the special characters that you'll encounter. If you are infinitely 
amused by this sort of thing, look at the Jargon File (http://catb. org/~esr/jargon/html/) or its printed 
companion, The New Hacker's Dictionary [Raymond]. 


Table 1-1 onthe next page lists a select set of the special characters, what they are, what people call 
them, and their uses. You have already seen many of these characters in this chapter. Not all meanings of 
each character are identified because there are too many to list. Some utilities, such as the Perl 
programming language, use nearly every one of these special characters! Also, keep in mind that these 


are the American names for the characters. 


Table 1-1: Special Characters 























Character Name(s) Uses 
i star Regular expression, wildcard character 
dot Current directory, file/hostname delimiter 
| bang Negation, command history 
| pipe Command pipes 
/ (forward) slash Directory delimiter, search command 
\ backslash Literals, macros (never directories) 
$ dollar Variable denotation, end of line 
l tick, (single) quote Literal strings 
backtick, backquote Command substitution 
i double quote Semi-literal strings 
. caret Negation, beginning of line 
aa tilde, squiggle Negation, directory shortcut 
# hash, sharp, pound Comments, preprocessor, substitutions 
[] (Square) brackets Ranges 
{\ (curly) braces Statement blocks, ranges 
underscore Cheap substitute for a space 





Note You will often see control characters marked with a caret; for example, ^c for CONTROL-C. 


1.11Command-Line Editing 


As you play around with the shell, you may notice that you can edit the command line with the left and 
right arrow keys, as well as page through previous commands with the up and down arrows. This is as 
good as standard on almost any Linux system. 


However, you should forget about the arrow keys and use control key sequences instead. If you learn and 
practice the ones listed in Table 1-2, you will have a great advantage when entering text in the many Unix 
programs that use these standard keystrokes. 


Table 1-2: Command-Line Keystrokes 


Keystroke Action 
CONT ROL-B Move cursor left 





CONT ROL-F | Move cursor right 
CONTROL-P View previous command (or move cursor up) 
CONT ROL-N View next command (or move cursor down) 


CONTROL-A Move the cursor to the beginning of the line 


CONT ROL-E | Move the cursor to the end of the line 


CONT ROL-W Erase the preceding word 
CONT ROL-U Erase the entire line 











1.12Text Editors 


Speaking of editing, it's time for you to learn an editor. To get serious with Unix, you must be able to edit 
text files without damaging them. Most parts of the system use plain-text configuration files (for example, 
those in /etc). It's not too difficult to edit files, but you will do it so often that you need a serious, powerful 
tool for the job. 


You should make a serious attempt to learn one of the two de facto standard Unix text editors, vi and 
emacs. Most Unix wizards are religious about their choice of editor. Don't listen to them. Instead, choose 
for yourself. If you choose an editor that matches your personality, you will find it easier to learn. 


a |f you want an editor that can do almost anything and has extensive online help, and you don't mind 
doing some extra typing to get these features, try emacs. 


a |f speed means everything to you, give vi a shot; it "plays" a bit like a video game. 


Learning the vi Editor [Lamb] can tell you everything you need to know about vi. For emacs, use the 
online tutorial: Start emacs from the shell prompt or a GUI menu, type CONTROL-H, and then type t. If 
you want a book, look at GNU Emacs Manual [Stallman]. 


You might be tempted to bumble around with a "friendlier" editor, such as pico or one of the myriad GUI 
editors when you first start out, but if you're the type of person who tends to make a habit out of the first 
thing that you use, you don't want to go down this route. 


Incidentally, the editing text is where you will first start to see a difference between the terminal and the 
GUI. Editors such as vi run inside the terminal window, using the standard terminal I/O interface that you 
are now starting to learn. However, GUI editors start their own window and present their own interface, 
independent of terminals. 


1.13Getting Online Help 


Linux systems come with a wealth of documentation. For basic commands, the manual pages (or man 
pages) tell you what you need to know. To access this manual, use the man command. For example, to 
see the manual page for the 1s command, run man as follows: 


man ls 


Most manual pages concentrate primarily on reference information. They may contain some examples and 
cross-references, but that's about it. Don't expect a tutorial, and don't expect an engaging literary style. For 
programs with many options, the manual page often lists the options in some systematic way (for example, 
alphabetical order). It won't bother to tell you what the important ones are. If you are patient, you can 
usually find what you need to know. If you're excessively impatient, ask a friend, or pay someone to be 
your friend so that you can ask them. 


To search for a manual page by keyword, use the -k option: 


man -k keyword 


This is helpful if you don't quite know the name of the command that you want. 


Note If you have any additional questions about any of the commands described in the previous 
sections, you can find the answers with the man command. 


Manual pages fall into numbered sections. When someone refers to a manual page, the section number 
appears in parentheses next to the name — ping(8), for example. Table 1-3 explains the section numbers: 


Table 1-3: Online Manual Sections 





Section | Description 


1 User commands 
Low-level system calls 
Higher-level Unix programming library documentation 


| Device interface and driver information 





File descriptions (system configuration files) 


Games 





| File formats, conventions, and encodings (ASCII, suffixes, and so on) 








CO N O | oO} | OW | Po 


| System commands and servers 





Sections 1, 5, 7, and 8 are good supplements to this book. Section 4 may be of marginal use. Section 6 
would be great if only it were a little larger. 


You can select a manual page by section. This is sometimes important because man displays the first 
manual page that it finds for a particular search term. For example, if you want to see the /etc/passwd 
file description (as opposed to the passwd command), you can insert the section number before the page 
name: 


man 5 passwd 


Manual pages cover the essentials, but there are many more ways to get online help. If you're just looking 
for a certain option for a command, try typing a command name followed by --hel1p or -h (the exact 
option varies from command to command). You may get a deluge (as in the case of 1s --help), but you 
may find just what you're looking for. 


Some time ago, the GNU Project decided that it didn't like manual pages very much and switched to 


another format called info (or texinfo). Often, this documentation goes further than a typical manual page, 
but it is sometimes more complex. To access an info page, use info with the command name: 


infocommand 


Some packages dump their available documentation into /usr/share/doc with no regard for online 
manual systems such as man or info. Have a look in this directory on your system if you find yourself 
searching for documentation. And as ever, don't hesitate to look for help on the Internet if you have a 

connection. 


1.14Shell Input and Output 


Now that you are familiar with basic Unix commands, files, and directories, you are ready learn the shell's 
I/O tricks. You can redirect the standard input and output. Let's start with standard output. 


If you wish to send the output of command to a file instead of the terminal, use the > redirection character: 


command > file 


The shell creates file if it does not already exist. If £i1e does exist, the shell erases the original file first; 


this is called clobbering the file. Some shells have parameters that prevent clobbering. For example, you 
can type set -C to avoid clobbering in bash. 


If you don't want to overwrite a file, you can append the output to the file instead with the >> redirection 
syntax: 


command >> file 


This is a handy way to collect output in one place when repeatedly executing a variant of the same 
command. 


To send the output of a command to the input of another command, use the pipe (|). To see how this 
works, try these two commands: 


head /proc/cpuinfo 
head /proc/cpuinfo | tr a-z A-Z 


You can send output through as many piped commands as you wish; just add another pipe (|) before each 
additional command. 


1.14.1Standard Error 


Occasionally, you may redirect standard output but find that the program still prints something on the 
terminal. This is standard error (stderr), an additional output stream for diagnostics and debugging. Try 
this command, which produces an error: 


Is JL firirirg: 2 £ 


After completion, £ should be empty, but you still see the following error message on the terminal as 
standard error: 


Ls; /fELLTEL LLL: No such file or directory 


You can redirect the standard error if you like. If you want to send standard output to £ and standard error 
toe, use the following command: 


Le yfirl L = £ 2S Se 


The number 2 specifies the stream ID that the shell modifies. Stream ID 1 is standard output (the default), 
and 2 is standard error. 


You can also send the standard error to the same place as stdout withthe >« notation. For example, to 
send both standard output and standard error to the file named f, try this command: 


is JLEEreitin > f 2>G) 


1.14.2Standard Input Redirection 


It is also possible to channel a file to a program's standard input with the < operator. Here's an example: 


head < /proc/cpuinfo 


You will occasionally run into a program that requires this sort of redirection. However, because most Unix 
commands accept filenames as arguments, this redirection isn't very common. For example, the 
preceding command could have been written as head /proc/cpuinfo. 


1.15Understanding Error Messages 


When you encounter a problem on a Unix-like system such as Linux, you must read the error message. 
Unlike messages from other operating systems, Unix errors usually tell you exactly what went wrong. 


Most Unix programs generate and report the same basic error messages, but there can be subtle 
differences between the output of any two programs. Here is an example that you'll certainly encounter in 
some form or other (the error message is in boldface): 


S ls /dsafsda 
ls: /dsafsda: No such file or directory 


There are three components to this message: 


a» The program name, 1s. Some programs omit this identifying information, which can be annoying 
when you are writing shell scripts, but it's not really a big deal. 


=» The filename, /dsafsda, whichis amore specific piece of information. There's some problem with 
this path. 


=» [he specific error, No such file or directory, indicates the problem with the filename. 


Putting it all together, you get something like "1s tried to open /dsafsda but couldn't because it does not 
exist." This may seem kind of obvious, but these messages can get a little confusing when you run a shell 
script that includes an erroneous command under a different name. 


Always address the first error first. For example, some programs report that they can't do something before 
reporting a host of other problems. For example, let's say you run a fictitious program called scumd, and 
you see this error message: 


scumd: /etc/scumd/config: No such file or directory 


Following this is a huge list of other error messages that looks like a complete catastrophe. Don't let those 
other errors distract you. You probably just need to create /etc/scumd/confidg. 


Note Do not confuse error messages with warning messages. Warnings often look like errors, except 
that they contain the word "warning." A warning usually means something is wrong but that the 
program will try to continue running anyway. To fix a problem noted in a warning message, you 
may have to hunt down a process and kill it before doing anything else (you'll read about 
processes in Section 1.16). 


1.15.1Common Errors 


Many of the errors that you will encounter in Unix programs result from things that can go wrong with files 
and processes. Here is the error message hit parade: 


No Such File or Directory 
This is the number one error. You tried to access a file that does not exist. Because the Unix file I/O 
system does not discriminate between files and directories, this error message occurs everywhere. You get 


it when you try to read a file that does not exist, when you try to change to a directory that isn't there, when 
you try to write to a file in a directory that does not exist, and so on. 


File Exists 


In this case, you probably tried to create a file that already exists. This is common when you try to create a 
directory with the same name as a file. 


Not a Directory, is a Directory 


These messages pop up when you try to use a file as a directory, or a directory asa file. You may have to 
hunt around a little after you see the error message. Here is an example: 


> Louch a 
S$ touch a/b 
touch:a/b: Not a directory 


Notice that the error message only applies to the a part of a/b. When you encounter this problem, you 
may need to dig around alittle to find the particular file that is being treated like a directory. 


No Space Left on Device 


You're out of disk space. See Section 2.4.7 for information on how to manage this. 


Permission Denied 


You get this error when you attempt to read or write to a file or directory that you're not allowed to (that is, 
you have insufficient access privileges). This includes the case when you try to execute a file that does not 
have the execute bit set (even if you can read the file). You will read more about permissions in Section 
1.17. 


Operation Not Permitted 


This usually happens when you try to kill a process that you don't own. 


Segmentation Fault, Bus Error 


A segmentation fault essentially means that the person who wrote the program that you just ran screwed 
up somewhere. The program tried to access some part of memory that it was not allowed to touch, and the 
operating system killed it. A bus error is similar, except that it tried to access some memory in a particular 
way that it shouldn't. When you get one of these errors, you might be giving a program some input that it 
did not expect. 


1.16Processes 
Aprocess is a running program. Each process on the system has a numeric process ID (PID). For a quick 


listing of the processes that you're running, just run the ps command on the command line. You should get 
a list like this: 


PID TTY STAT TIME COMMAND 


520 pU 5 0:00 -bash 

545 os 3:59 /usr/X11R6/bin/ctwm -W 

548 7-5 0:10 xclock -geometry -0-0O 
2159 pd SW 0:00 /usr/bin/vi lib/addresses 
51956: “po EB 0200 ps 


This is an abridged listing; yours will be much longer if you're running a windowing system. The fields are 
as follows: 


= PID The process ID. 
m TTY The terminal device where the process is running (don't worry about this for now). 


=» STAT The process status; that is, what the process is doing at the given time and where its memory 
resides. For example, S means sleeping and R means running. Check the ps(1) manual page for all 
the symbols. 


= TIME The amount of CPU time (in minutes and seconds) that the process has used so far. In other 
words, this is the total amount of time that the process has spent running instructions on the 
processor. 


= COMMAND This one might seem obvious, but be aware that a process can change this field from its 
original value. 


If you're interested in all processes on the system (not just the ones you're running), use ps ax, and if 
you'd like a more detailed report on process characteristics, use ps u. As with other programs, you can 
combine options, asin ps aux. Another important option that works in conjunction with a is w, which tells 
ps to print the full command name if it's too long to fit on one line. 


To check on a specific process, add its PID to the argument list of the ps command. For example, one 
way to inspect the current shell process is with ps u $$ ($$ isa shell variable that evaluates to the 
current shell's PID). 


You'll find information on two handy administration commands called top and 1sof in Section 4.8. These 
can be useful when locating processes, even when doing something other than system maintenance. 


1.16.1Killing Processes 
To terminate a process, send it a signa/ with the kill command. In most cases, all you need to do is this: 


killpid 


There are many types of signals. The default signal is TERM, or terminate. You can send different signals 
by adding an extra option to kill. Let's say that you don't want to terminate the process, but rather, 
freeze it with the STOP signal. Use this command: 


kill -STOP pid 


A process stopped in this manner is still in memory, ready to pick up where it left off. Use the CONT signal 
to set the process on its way again: 


kill -CONT pid 


As you may recall from Section 1.2, CONTROL-C terminates a process running in the current terminal. 
This is the same as using kill to end the process with the INT (interrupt) signal. 


The most brutal way to terminate a process is with the KILL signal. Other signals give the process a 
chance to clean up after itself, but KILL does not — the operating system terminates the process and 
forcibly removes it from memory. Use this as a last resort. 


Warning Don't kill processes indiscriminately, especially if you don't know what they're doing. This can 
be akin to shooting yourself in the foot. 


1.16.2Job Control 


Shells also support job control, away to send STOP and CONT signals to programs by using funny 
keystrokes and commands. For example, you can send a STOP signal with CONT ROL-Z and start the 
process again by typing fg or bg. 


Despite the habits of many experienced users, job control is not necessary and can be confusing to 
beginners — it is not uncommon for the user to press CONT ROL-Z instead of CONTROL-C and have 
many suspended processes sitting about. 


If you want to run multiple shell programs, you can either run each program in a separate terminal window, 
put non-interactive processes in the background (explained in the next section), or learn to use the 
screen program. 


1.16.3Background Processes 


Normally, when you run a Unix command from the shell, you do not get the shell prompt back until the 
program has finished executing. However, you can detach a process from the shell and put it in the 
"background" with the ampersand («); this gives you the prompt back. For example, if you have a large file 
that you need to decompress with gunzip (you'll see this in Section 1.18), and you want to do some other 
stuff while it's running, run the command as in this gunzip example: 


gunzipfile.gz & 


The shell responds by printing the PID of the new background process, and the prompt returns 
immediately so that you can continue working. The process also continues to run after you log out, which 
comes in particularly handy if you have a program that does a lot of number crunching that you need to 
run for a while. Depending on your setup, the shell may notify you when the process completes. 


The dark side to background processes is that they may expect to work with the standard input (or worse). 
If the program wants to read something from the standard input when it is in the background, it can freeze 
(try £g to bring it back), or it may terminate. If the program writes to the standard output or standard error, 
the output can appear in the terminal window with no regard to anything else running there. 


The best way to make sure that a background process doesn't bother you is to redirect its output (and 
possibly input) as described in Section 1.14. 


If spurious output from background processes gets in your way, you should know how to redraw the 
content of your terminal window. bash and most full-screen interactive programs support CONTROL-L to 
redraw the entire screen. If a program is reading from the standard input, CONTROL-R usually redraws the 
current line. Unfortunately, pressing the wrong sequence in the wrong situation can leave you in an even 
worse situation than before. For example, CONTROL-R at the bash prompt puts you in an annoying mode 
called reverse isearch. 


1.17File Modes and Permissions 


Every Unix file has a set of permissions that determine whether you can read, write, or run the file. 
Runningls -1 displays the permissions. Here's an example of such a display: 


—-rw-r--r-- 1 juser somegroup 7041 Mar 26 19:34 endnotes.html 


The first column represents the mode of the file, and it is shown in bold. The mode represents the file's 
permissions and some extra information. There are four parts to the mode; Figure 1-1 on the next page 
illustrates the pieces. The first character of the mode is the file type. A dash (-) in this position, as in the 
example, denotes a regular file, meaning that there is nothing special about it. This is by far the most 
common kind of file. Directories are also common, carrying d in the file type slot. The rest of the file types 
are listed in Section 2.3, but you don't need to know about them just yet. 


User permissions 
Type Group permissions 
| [Other permissions 
-rw-r--r-- 
Figure 1-1: The pieces of a file mode 


The rest of a file's mode contains the permissions, which break down into three sets: the user,group, and 
other permissions, in that order. For example, the rw- characters in the example are the user permissions, 
r-— are the group permissions, and r—- are the other permissions. 


Four basic things can appear in each permission set: 
a r Means that the file is readable 
a w Means that the file is writable 
=» x Means that the file is executable (that is, you can run it as a program) 
a - Means nothing 


The user permissions (the first set) pertain to the user who owns the file. In the preceding example, that's 
juser. 


The second set, group permissions, are for the file's group (somegroup in the example). Any user in that 
group can take advantage of these permissions. (Use the groups command to see what group you're in, 
and see Section 4.3.2 for more information.) 


Everyone else on the system has access according to the other permissions. These are sometimes called 
world permissions. 


Note Each read, write, and execute permission slot is sometimes called a permission bit. Therefore, 
you may hear people refer to parts of the permissions as "the read bits." 


Some executable files have an s in the user permissions listing instead of an x. This indicates that the 
executable is setuid, meaning that when you execute the program, it runs as the file owner instead of you. 
Many programs use this setuid bit to run as root to get the special privileges they need to change system 
files. One example is the passwd program, which needs to change the /etc/passwd file. 


1.17.1Modifying Permissions 


To change permissions, use the chmod command. First, pick the set of permissions that you want to 
change, and then pick the bit to change. For example, say that you want to add group (g) and world (o, for 
"other") read (r) permissions to file. You can do it with two commands: 


chmodgtrfile 
chmodo+trfile 


Or you can do it all in one shot like this: 


chmodgotrfile 


To remove these permissions, use go-r instead of go+r. 


Note Obviously, you shouldn't make your files world writable because it gives anyone on your system 
the ability to change them. But would this give anyone connected to the Internet a chance to 
change your files? Probably not, unless your system has a network security hole. In that case, 
file permissions aren't going to help you. 


You may sometimes see people changing permissions with numbers, for example: 


chmod 644 file 


This is called an absolute change, because it sets all of the permission bits at once. To understand how 
this works, you need to know how to represent the permission bits in octal form (each numeral represents 
an octal number and corresponds to a permission set). If you're curious about this, look at the chmod(1) 
manual page or the info page. 


It is actually not that important to know how to construct absolute modes; it is easier to just memorize the 
modes that you use. Table 1-4 lists the most common absolute modes. 


Table 1-4: Absolute Permission Modes 


Mode Meaning Used For 

644 user: read/write; group, other: read files 

600 user: read/write; group, other: none files 

755 user: read/write/execute; group, other: read/execute directories, programs 


700 | user: read/write/execute; group, other: none iaoi programs 


711 user: read/write/execute; group, other: execute directories 


Directories also have permissions, as you can see from the preceding list. You can list the contents of a 
directory if the directory is readable, but you can only access a file inside if the directory is executable. 
One common mistake people make when setting the permissions of directories is to accidentally remove 
the execute bit when using absolute modes. 
















Finally, you can specify a set of default permissions with the umask shell command that are applied to any 
new file you create. Without going into too much detail, use umask 022 if you want everyone to be able to 
see all of the files and directories that you make, and use umask 077 if you don't. You need to put the 
umask command with the desired mode in one of your startup files to make your new default permissions 
apply to later sessions (see Chapter 16). 


1.17.2Symbolic Links 


Asymbolic link is a file that points to another file or a directory, effectively creating an alias. Itis similar to 
a shortcut in Windows. Symbolic links offer a quick way to provide access to an obscure directory path. 


In a long directory listing, these links look like this (notice the 1 as the file type in the file mode): 


lrwxrwxrwx 1 ruser users 11 Feb 27 13:52 funk -> /home/skunk 


If you try to access funk in this directory, the system gives you /home/skunk instead. Symbolic links are 
nothing more than names that point to other names. 


The names of the symbolic links and the paths to which they point don't actually have to mean anything. In 
the preceding example, /home/skunk doesn't need to exist. If /home/skunk does not in fact exist, any 
program that accesses funk just reports that funk doesn't exist (except for 1s funk, acommand that 
stupidly informs you that funk is funk). This can be baffling, because you can see something named 


funk right there in front of you. 


This is not the only way that symbolic links can be confusing. Another problem is that you cannot identify 
the characteristics of a link target just by looking at the name of the link. You must follow the link to find 
out if it goes to a file or directory. Your system may also have links that point to other links, which are 
calledchained symbolic links. 


To create a symbolic link from target to linkname, use this command: 


ln -s target linkname 


Thelinkname argument is the name of the symbolic link, the target argument is the path of the file or 
directory that the link points to, and the -s flag specifies a symbolic link (see the warning on the next 


page). 


When making a symbolic link, check the command twice before you run it, because there are a number of 
things that can go wrong. For example, if you reverse the order of the arguments (ln -s linkname 
target) you're infor some fun when linkname is a directory that already exists. In this case, 1n creates 
a link named targetinsidelinkname (the link points to itself unless 1 inkname is a full path). Therefore, 
if something goes wrong when you create a symbolic link to a directory, check that directory for errant 
symbolic links and remove them. 


Symbolic links can also cause headaches when you're not aware of their presence. For example, you can 
easily edit what you think is a copy of a file but is actually a symbolic link to the original. 


Warning Don't forget the -s option when creating a symbolic link. Without it, 1n creates a hard link, 
giving an additional real filename to a single file. The new filename has all of the status of 
the old one; it points (links) directly to the file data instead of to another filename, asa 
symbolic link does. Hard links can be even more confusing than symbolic links. Avoid them. 


With all of these warnings regarding symbolic links, you may wonder why anyone ever bothers with them. 
The simple reason is that they offer a convenient way to share files and patch up small problems. 


1.18Archiving and Compressing Files 


Now that you know all about files, permissions, and the errors that you might get, you need to master tar 
andgzip. 


Start with gzip (GNU Zip), the current standard Unix compression program. A file that ends with .gz isa 
GNU Zip archive. Use gunzipfile.gz touncompress file.gz and remove the suffix; to compress it 
again, use gzipfile. 


gzip does not perform the additional function of archiving, like the ZIP programs for other operating 
systems — that is, it does not pack multiple files and directories into one file, thus creating an archive. To 
create an archive, use tar instead: 


tar cvf archive.tarfilel file2 


tar archives have a .tar Suffix. Inthe preceding example, filel,file2, and so on are the names of 
the files and directories that you wish to place in the archive named archive.tar. The c flag denotes 
create mode. You will learn about the v and £f flags later in this section. 


You can also unpack a .tar file with tar: 


tar xvf file.tar 


Study this command. The xvf part specifies the mode and options; it is the same as -xvf. The meanings 
are as follows: 


a» Thex flag puts tar into extract (unpack) mode. You can extract individual parts of the archive by 
placing the names of the parts at the end of the command line, but you must know their exact names. 
To find out for sure, see the table of contents mode described shortly. 


= Thev flag activates verbose diagnostic output, causing tar to print the names of the files and 
directories in the archive when it encounters them. Adding another v causes tar to print details such 
as file size and permissions. 


=» [hef flag denotes the file option. The next argument on the command line must be the file on which 
tar is to work (in the preceding example, it is file.tar). You must use this option and a filename at 
all times — there is one exception, but it involves tape drives (see Section 13.6). To use standard 
input or output, use - instead of the filename. 


Note When using extract mode, remember that tar does not remove the archived . tar file after 
extracting its contents. 


Before unpacking, it's usually a good idea to check the contents of a .tar file with the t (table of 
contents) mode instead of x. This mode verifies the archive's basic integrity and prints the names of all 
files inside. 


The most important reason for testing an archive is that sometimes unpacking an archive can dump a 
huge mess of files into the current directory. This is difficult to clean up. When you check an archive with 
thet mode, verify that everything is in a rational directory structure — that Is, all file pathnames in the 
archive should start with the same directory. If you're not sure, create a temporary directory, change to 
that, and then extract. You can always use mv * .. if it turned out to be a false alarm. 


One last significant option to tar is p, which preserves permissions. Use this in extract mode to override 
yourumask and get the exact permissions specified in the archive. The p option is the default when 
working as the superuser. If you're having trouble with permissions and ownership when unpacking an 
archive as the superuser, make sure that you are waiting until the command terminates and you get the 
shell prompt back. Although you may only want to extract a small part of the archive, tar must run 
through the whole thing. You must not interrupt the process, because it sets the permissions only after 
checking the entire archive. 


You should commit all of the tar options and modes in this section to memory; know them cold. If you're 
having trouble, make some flash cards. This may sound like a grade-school strategy, but it is very 


important to avoid careless mistakes with this command. 


1.18.1Compressed Archives (.tar.gz) 


Many beginners find it confusing that archives normally come compressed, where the archive file ends in 
.tar.gz. To unpack a compressed archive, work from the right side to the left; get rid of the . gz first and 
then worry about the .t ar. For example, these two commands decompress and unpack file.tar.gz: 


Gunziprite.tar, gz 
tar xvf file.tar 


When you're starting out, it's fine to do this one step at a time, first running gunzip to decompress and 
thentar to verify and unpack. When you do it enough, you soon memorize the entire archiving and 
compression process. However, this is not the fastest or most efficient way to invoke tar ona 
compressed archive. In particular, it wastes system resources — disk space and kernel I/O time. 


You can combine archival and compression functions with a pipeline; for example, this command pipeline 
unpacksfile.tar.gz: 


zcatfile.tar.gz | tar xvf - 


zcat Is the same as gunzip -—dc. The -qa option decompresses and the -c option sends the result to 
standard output (in this case, to the tar command). 


Because this is such acommon operation, the version of tar that comes with Linux has a shortcut. You 
can use z as an option to automatically invoke gzip onthe archive. For example, use tar ztvf 
file.tar.gz to verify a compressed archive. However, for the sake of learning, you should make an 
effort to master the longer form before taking the shortcut. 


Note A. tgz file is the same as a . tar.gz file. The name is meant to fit into FAT (MS-DOS based) 
filesystems. 


1.18.2O0ther Compression Utilities 


A newer compression program gaining some popularity in Unix is bzip2, where compressed files end with 
.bz2. Marginally slower than gzip, it often compacts text files a little more, and is therefore increasingly 
popular in the distribution of source code. The decompressing program is bunzip2, and the options of 
both components are close enough to those of gzip that you don't need to learn anything new. 


Most Linux distributions come with zip and unzip programs compatible with the ZIP archives on 
Windows systems. They work on the usual . zip files as well as self-extracting archives ending in .exe. 


If you encounter a file that ends in . Zz, you have found a relic created by the compress program, once the 
Unix standard. gunzip can unpack these files, but gzip will not create them. 


Chapter 2: Devices, Disks, Filesystems, and the Kernel 


This chapter is a guided tour of the infrastructure in a functioning Linux system, including the system 
directories, devices, and the kernel. A large part of the material in this chapter deals with filesystems. 
When you master the workings of Linux filesystems, you will find it much easier to deal with system 
crashes, manage software, and accommodate new hardware. 


Note You must do most system configuration and maintenance work as the superuser; this and all 
remaining chapters assume that you have this capability. 


2.1 Directory Hierarchy 


As mentioned in Chapter 1, your Linux system has a root directory (/). The root contains several 
subdirectories and possibly a file or two. Keep this area clean; don't let stray files end up here. But don't 
fall victim to a syndrome that affects many administrators — removing files in a zealous attempt to keep 
the system "clean." If you don't know what something does (and can't figure it out), don't remove it. 


The names of the directories in / have special meanings that pervade the Linux system in many other 
subdirectories. For example, there is a /bin directory in the root, but there are also many other bin 
directories throughout the system, including /usr/bin and /usr/local/bin. All of these bin 
directories contain executables. 


If you want to know the gory details of the many directories on the system, have a look at the Filesystem 
Hierarchy Standard [Filesystem]. For the most part, the directory names reflect the root and /usr directory 
patterns described in the next sections. 


2.1.1The Essential Root Subdirectories 


The most important of the root directory's subdirectories are the following: 


=» bin Contains binary programs (also Known as an executables), which are programs that are ready to 
run. Most of the basic Unix commands such as 1s and cp arein /bin. However, some of the files in 
/bin are not in binary format because shell scripts perform the tasks of binaries in modern systems. 


= dev Contains device files. Read more about these special files in Section 2.3. 


=» etc The core system configuration directory (pronounced EHT -cee). User password, boot, device, 
networking, and other setup files are here. Many items in /etc are specific to the particular hardware 
on the machine — for example, the /etc/X11 directory contains the graphics card configuration. 


=» home Holds personal directories for normal users on the system. Most Unix installations conform to 
this standard. 


a» lib An abbreviation for library. In Linux, this directory holds library files containing code that 
executables can use. There are two types of libraries: static and shared. The /1ib directory should 
contain only shared libraries, but other 1ib directories such as /usr/1ib contain both varieties, as 
well as other auxiliary files. 


= proc Provides system statistics through a directory-and-file interface that you can browse with 
standard Unix tools, like those introduced in Chapter 1. Much of the /proc subdirectory structure on 
Linux is unique, but many other Unix variants have similar features. 


=» sbin The place to find system executables. Programs in sbin directories pertain to system 
management, so regular users usually do not have sbin components in their command paths. Many 
of the utilities don't work for normal users. 


=» tmp The place to put smaller temporary files that you don't care much about. Any user may read to 
and write from /tmp, but they may not have permission to access another user's files there. Some 


programs use this directory as a workspace. If something is extremely important, don't put itin /tmp. 
Most distributions clear /tmp when the machine boots, and some even remove its old files 
periodically. Don't fill /tmp either, because its space is usually shared with something critical (like the 
rest of /, for example). 


=» usr Pronounced as "user," but this subdirectory does not contain user files (there have been no user 
files in /usr on Unix systems long before Linux existed). Instead, /usr is a large directory hierarchy 
that looks a little like the root. The bulk of the Linux system resides in /usr. Many of the directory 
names in /usr are the same as in the root and hold the same type of files; /usr/bin and /usr/lib 
are two examples. The primary reason that the root does not contain the complete system is to keep 
space requirements low. The /usr directory is so important that Section 2.1.3 is dedicated to it. 


= var The "variable" subdirectory, where programs record runtime information. System logging, user 
tracking, caches, and other files that system programs create and tend all go into /var. Thereisa 
/var/tmp similar to /tmp, but the system doesn't wipe it clean on boot. 


2.1.2O0ther Root Subdirectories 


There are a few other interesting subdirectories in the root: 


=» boot Contains kernel boot loader files. These files pertain only to the very first stage of the Linux 
startup process; you will not find information about how Linux starts up its services in this directory. 


=» cdrom Where most Linux distributions attach a CD drive; the contents of the disc are under this 
directory. The root may contain similar directories for other removable devices. 


=» opt May contain additional third-party software. Many systems don't use /opt. 


2.1.3The /usr Directory 
The/usr directory may look relatively clean at first glance, but a quick look at /usr/binand /usr/lib 


reveals that there's a lot here. In addition to the subdirectories that you just read about, /usr contains the 
following: 


= include Holds header files used by the C compiler (see Section 8.1.2). 
=» info Contains GNU info pages (see Section 1.13). 


m local Where administrators can install their own software. Its structure should look like that of / and 
/usr. 


=» man Contains manual pages. Unformatted pages go in the man* directories here (according to the 
numbered category described in Section 1.13), and human-readable formatted pages go in cat*. 


=» share Contains files that should work on other kinds of Unix machines with no loss of functionality. 
That's the theory, anyway; a true share directory is becoming rare because there are no space issues 
on modern disks. It is often more of a pain to maintain a share directory than it's worth. In any case, 
man,info, and some other subdirectories are often found here. 


=m X11R6 Holds the core Linux windowing system software, called XFree86. However, the custom 
configuration files for your computer are usually in /etc/X11. 


m dict An oddball subdirectory containing dictionaries; /usr/dict /words Is one such dictionary file. 


2.2 The Kernel 


You now know what the Linux directory structure looks like. Before going on to devices and filesystems, 
you must learn some higher-level concepts about the kernel. 


Thekernel of an operating system is the very core of a running system. It's a very special program that 
manages all processes, device drivers, and I/O. When the system boots, the kernel runs first, initializing 
hardware and internal data structures. After the kernel completes this stage, it loads and starts the init 
program. 


The kernel also has many functions once the system boots. Process and device management are the most 
important. The idea is that although computers with one CPU run only one program at atime, the kernel 
can keep several programs in memory at the same time. Process and device management run together in 
a cycle like this: 


1. The kernel has control of the processor. There are several processes in memory. 
. The kernel selects a process and finds out where that process was last running. 


. The kernel relinquishes control of the processor to that process. 


. When the time is up, a clock interrupt stops the process and gives control back to the kernel. 


2 

3 

4. That process runs for a few microseconds. 

5 

6. The kernel takes care of any system work that it needs to do, such as reading to and from devices. 
7 


. Go to step 2. 


This description of the kernel's work is somewhat simplified, but as you can see, there is nothing magic 
about the kernel. It is not a process; it is just a piece of code that runs every now and then between 
processes. 


On Linux systems, the kernel is normally in a file called /vmlinuz or /boot/vmlinuz. A boot loader 
loads this file into memory and sets it in motion when the system boots. Details on to how to configure the 
boot loader and how to create a Linux kernel are in Chapters 4 and 10. 


If you are interested in the detailed workings of a kernel, the classic textbook is Operating System 
Concepts [Silberschatz]. 


2.3 Devices 


You will find it very easy to manipulate devices on a Unix system because the kernel normally presents 
the device I/O interface to system and user processes as files. Not only can a programmer use regular file 
operations to work with a device, but some devices are also accessible to standard programs like cat, so 
you don't have to be a programmer to use a device. Linux uses the same design of device files as other 
Unix flavors, but device filenames and functionality vary among flavors. 


Linix device files are inthe /dev directory, and running 1s /dev reveals that there are more than a few 
files in /dev. So how do you work with devices? 


To get started, consider this command: 


echo blah blah > /dev/null 


Like any command with redirected output, this sends some stuff on the standard output to a file. However, 
the file is /dev/nu11, a device, and the kernel decides what to do with any data written to this device. In 
the case of /dev/nu11, the kernel simply ignores the data (making /dev/nu11 a bitbucket). Of course, 
other devices actually do things, such as /dev/dsp, which plays a sound if you send appropriate output 
there (for example, cat blab.wav > /dev/dsp). 


To identify a device, use 1s -1 to geta long listing and look at the permissions. Here are four examples: 


Dru=ry -=== 1 root disk oy 65 Jul 20 1998 hdbl 
Crw-rw-rw- 1 root roort T 3. dal 20 1998 nul: 
pro-r-=I- 1 root root O Mar 3 19:17 gpmdata 
Srw-rw-rw- l root root O Dec 18 07:43 log 


Notice the very first character of each line in the listing. If this character is b,c,p, or s, then the file is a 
device. These letters stand for block,character,pipe, and socket, respectively: 


=» Block device Programs access data in a block device in fixed chunks. The hdb1 in the preceding 
example is a disk device. Because disks break down into blocks of data, itis only natural that a disk 
be a block device. A block device's total size is fixed, and a program has random access to any block 
in the device. 


= Character deviceCharacter devices work with data streams. You can only read characters from or 
write characters to these devices, like /dev/nu11 in the previous example. Character devices don't 
have a size; when you read from or write to a character device, the kernel usually performs a read or 
write operation on the device, leaving no record of the activity inside the kernel. Printers are character 
devices, and after the kernel sends data to a printer, the responsibility for that data passes to the 
printer; the kernel cannot back up and reexamine the data stream. 


=» Pipe deviceNamed pipes are like character devices, but there is another process at the other end of 
the I/O stream instead of a kernel driver. An example of such a process is gpm, a program that can 
duplicate mouse events and send them to the named pipe /dev/gpmdata for use by other programs. 


= Socket deviceSockets are special-purpose |/O files offering a type of network interface. For 
example, the gpm program accepts control commands through the /dev/gpmct1 socket. You don't 
need to worry about sockets if you're not a network programmer. 


The numbers before the dates in the first two lines of the previous listing are the major and minor device 


numbers that help the kernel identify the device. Similar devices usually have the same major number, 
such as hda3 and hdb1 (these are both hard disk partitions). 


2.3.1dd and Devices 


The name dd stands for convert and copy. dd is extremely useful when working with block and character 


devices. This program's sole function is to read from an input file or stream and write to an output file or 
stream, possibly doing some encoding conversion on the way. It was originally developed for reblocking. 


dd copies data in blocks of a fixed size. Here is an example of using dd with a character device and some 
common options: 


dd if=/dev/zero of=new file bs=1024 count=1 


As you can see, the dd option format is different than the option formats of most other Unix commands; 
rather than use the - character to signal an option, you name an option and set its value to something with 
the= sign. The preceding example copies a single 1024 -byte block from /dev/zero (a continuous 
stream of zero bytes) to the file new_file. These are the important dd options: 


m if=file The input file. The default is the standard input. 
=m of=file The output file. The default is the standard output. 


=m bs=size The block size; dd reads and writes this many bytes of data at atime. To abbreviate large 
chunks of data, you may use b and k to signify 512 and 1024 bytes. Therefore, the example above 
could read bs=1k instead of bs=1024. 


m ibs=size, obs=size [he input and output block sizes. If you can use the same block size for both 
input and output, use the bs option, but if this is impossible, then use ibs and obs for input and 
output, respectively. 


= count=num The total number of blocks to copy. When working with a huge file, or with a device that 
supplies an endless stream of data (like /dev/zero), you want dd to stop at a fixed point. Otherwise 
you could waste a lot of disk space, CPU time, or both. count can be used in conjunction with the 
skip parameter to copy a small piece out of a large file or device. 


m skip=num Skip past the first num blocks in the input file or stream; do not copy them to the output. 


2.3.2Device Name Summary 


Sometimes it is difficult to find the name of a device (for example, when partitioning a disk). Here are two 
tactics for finding out: 


=» Guess the name from the output of the dmesg command (which prints out the last few kernel 
messages) or the kernel system log file (see Section 4.1); this output might contain a description of 
the devices on your system. 


m Runcat /proc/devices to see the block and character devices for which your system currently 
has drivers. Each line consists of a number and name. The number is the major number of the device 
described in Section 2.3. If you can guess the device from the name, look in /dev for the character or 
block devices with the corresponding major number, and you've found the device files. 


Neither method is terribly reliable, especially because the kernel does not load certain device drivers until 
you try to use them. (Section 10.8 has information on the driver-loading mechanism. ) 


The following sections list the most common Linux devices and their naming conventions. 


Hard Disks: /dev/hd* 
These are the ATA (IDE) disks and partitions. All are block devices. 


Two example names are /dev/hdal and /dev/hdb. The letter after hd identifies the disk, and the 
number represents the partition. A device without a number is a device for an entire disk. (Information on 
how to partition disks is in Section 2.3.4. To connect disks to your current systems, start at Section 2.4.3.) 


SCSI Disks: /dev/sd* 


SCSI disks carry the names /dev/sda,/dev/sdb, and so on, and the disks work much like their ATA 
counterparts, although the SCSI disk names do not directly correspond to a SCSI host controller and 
target. Linux assigns the devices in the order that it encounters the disks. 


For example, if you have two SCSI controllers, scsi0 and scsil, with disks at scsi0O targets 0, 3, and 


scsil target 1, the device assignments are as shown in Table 2-1. 


Table 2-1: Sample Device Assignments 


Controller Target Device Assignment 


scsi) 0 eee saa 


SCs i.0 3 /dev/sdb 
Scat | 1 /dev/sdc 














This naming scheme can cause problems when reconfiguring hardware. Let's say that scsi0 target 3 
explodes and you must remove the disk so that the machine can work again. When you do so, scsil 
target 1 moves to /dev/sdb from /dev/sdc, and you have to change the fstab file (described later in 
this chapter). 


Terminals: /dev/tty*, /dev/pts/*, /dev/tty 


Terminals are devices for moving characters between the system and an I/O device, usually for text output 
to a terminal screen. The terminal device interface goes back a long way, to the days when terminals were 
typewriter-based devices. 


Pseudo-terminal devices are emulated terminals that understand the I/O features of real terminals, but 
rather than talk to a real piece of hardware, the kernel presents the I/O interface to a piece of software, 
such as a shell window. 


Two common terminal devices are /dev/tty1 (the first virtual console) and /dev/pts/0 (the first 
pseudo-terminal device). 


The/dev/tty device is the controlling terminal of the current process. If a program is currently reading 
from and writing to a terminal, this device is a synonym for that terminal. A process does not need to be 
attached to aterminal. 


Serial Ports: /dev/ttyS* 


Serial ports are special terminal devices. You can't do much on the command line with serial port devices 
because there are too many settings to worry about, such as baud rate and flow control. 


The port known as COM1 on Windows is /dev/ttyS0, COM2 is /dev/ttyS1, and so on. For add-in 
modem cards, check the output of the dmesg command for the port assignment. 


Floppy Disks: /dev/fd* 


Section 11.1.1 covers operations on these block devices. The 3.5-inch floppy on most modern systems is 
/dev/fd0. 


Parallel Ports: /dev/Ip0, /dev/Ip1 


These unidirectional port devices correspond to LPT 1 and LPT2 in Windows. You can send files (Such as a 
file to be printed) directly to a parallel port with the cat command, but you might need to give the printer 
an extra form feed or reset afterward. 


The bidirectional parallel ports are /dev/parport0O and /dev/parportl. 


Audio Devices: /dev/dsp, /dev/audio, /dev/mixer, /dev/snd/", etc. 


Linux has two different sets of audio devices. There are separate devices for the OSS (Open Sound 
System) and the newer ALSA (Advanced Linux Sound Architecture) system interface. Linux systems that 
use ALSA usually contain OSS compatibility devices, because most applications still use OSS. Some 
rudimentary operations are possible with the dsp and audio devices that belong to OSS. As mentioned 
earlier, the computer plays any WAV file that you send to /dev/dsp. However, it may not sound right due 
to frequency mismatches. The ALSA devices are in the /dev/snd directory, but you can't do much by 
redirecting standard I/O to them. 


Note Theplay and aplay programs can play samples from the command line. To adjust the volume 
and mixer settings, aumix and alsamixer are available on most systems. 


2.3.3Creating Device Files 


To create one individual device file, use mknod. You must know the device name as well as its major and 
minor numbers. For example, if you remove /dev/hda2 by accident, you can create it again with this 
command: 


mknod /dev/hda2 b 3 2 


Theb32 specifies a block device with a major number 3 and a minor number 2. For character or named 
pipe devices, use c or p instead of b. 


Themknod command is useful only for creating the occasional missing device or named pipe. As you 
upgrade your system and add device drivers, you may need to create entirely new groups of devices. 
Because there are so many devices, it's better to use the MAKEDEV program (found in /dev) to create 
groups of devices. Device groups can be named after the common part of several device names, such as 
hda, or they can have a completely separate name, such as std-hd. For example, to create all devices 
beginning with hda, run this command: 


/dev/MAKEDEV hda 


Note TheMAKEDEV command is harmless if the devices already exist on your system. 


devfs 


Before you go to the trouble of making any device files, you should see whether you're running devfs, an 
automatic device-file generation system. The easiest way to check for devfs isto run mount and look for 
devfs in the output. 


If your system runs devfs, you should not have to create missing device files because the kernel 
maintains the directory of available device files. Linux kernels typically configure devfs at boot time, 
starting a special auxiliary system program called devfsd that gives the administrator the ability to 
customize the device system. 


Note devfs was an experimental feature in older Linux kernels, but is classified as obsolete in the 
latest Linux kernels. 
2.3.4Partitioning Disk Devices 
Before you can use a new disk on your system, you need to know how to partition disks with their device 
files. If you've never worked with partitions before, don't worry. A disk partition is just a piece of the disk 
dedicated to one purpose, such as a specific directory tree. On PCs, partitions have numbers starting at 1. 
To get started, identify the disk device that you want to partition. Most PCs have two ATA interfaces, 


called primary and secondary interfaces, and each interface can have a master and slave disk, for a total 
of four disk devices. The standard ATA disk assignments are as follows: 


u /dev/hda Master disk, primary interface 
a /dev/hdb Slave disk, primary interface 
a /dev/hdc Master disk, secondary interface 


a /dev/hdd Slave disk, secondary interface 


There are several partitioning utilities, but the most simple and direct is fdisk. To get started, run fdisk 
dev (where dev Is one of the devices listed above), then print the current partition list with p. 


Here is sample from fdisk output for a device with three partitions, two containing filesystems and one 
with swap (see Section 2.5 for more information on swap partitions). 


Disk /dev/hda: 240 heads, 63 sectors, 2584 cylinders 
Units = cylinders of 15120 * 512 bytes 


Device Boot obag End Blocks Id System 
/dev/hdal ih L36 POZSAZS 2 EINU 
/dev/hda2 Ae ay 204 514080 82 Linux swap 
/dev/hda3 205 2504 17992800 83 Linux 


Partition dimensions are usually in units of cylinders. Each partition has a start and end cylinder, 
determining its size, but the amount of space per cylinder varies depending on the disk. The second line in 
the preceding output shows you the cylinder size, but you don't need to do any weird computations to 
create partitions. 


When creating a new partition, you only need to know the starting cylinder. Cylinders do not overlap in PC 
partitions, so if you're partitioning a disk, pick the first available cylinder in the fdisk partition list output. 
In the preceding example, you would choose 2585 as the starting cylinder. 


Each partition also has a system ID, a number that represents an operating system. Linux uses 83 for 
partitions containing files and 82 for Linux swap. 


When you partition a new disk, there is usually one partition on the disk already, containing some sort of 
variant on the FAT filesystem for Microsoft systems. If you want only a single partition on the disk, just 


change the system ID to Linux with the t command inside fdisk. However, if you want to customize the 
partitions, use d to delete any old partitions and n to add new ones. 


fdisk Is easy to use because each command steps you through the partition number and size. In 
addition, there is an important safety feature: fdisk does not actually change the partition table until you 
tell it to with the w command. If you're uneasy about your changes, or you were just testing something, use 
q to exit without altering the disk. 


Here is an example of fdisk in action on a very small disk. Command input is in boldface: 
#£idisk /dev/hdc 
Command (m for help): p 


Disk /dev/hdc: 2 heads, 16 sectors, 247 cylinders 
Units = cylinders of 32 * 512 bytes 


Device Boot Seal End Blocks Id System 
/dev/hdcl * 1 494 7891+ 1 FAT12 


Command (m for help): d 


Partition number (1-4): 1 
Command (m for help): n 
Command action 

e extended 

p Primary- partition. (1-4) 
P 
Partition number (l-4): 1 


Last cylinder or tsize Or tsizeM or tsizek (lez4a7, default 2471. 120 
Command (m for help): t 

Partition number (1-4): 1 

Hex code (type L to List Codes) 83 


Command (m for help): p 


Disk /dev/hdc: 2 heads, 16 sectors, 247 cylinders 
Units = cylinders of 32 ~= pI- bytes 


Device Boot Stale End Blocks Id System 
/dev/hdcl 1 1:20 i ee So. -LINUX 


Command (m for help): w 


The partition table has been altered! 
Calling ioctl() to re-read partition table. 


WARNING: If you have created or modified any DOS 6.x 
partitions, please see the fdisk manual page for additional 
Informacion. 

Syncing disks. 


When you write the partition table to the disk with the w command, fdisktells the kernel to re-read the 
partition table from the disk and update the system's in-memory partition list (fdisk does not relay the 
new partition list directly to the kernel). Therefore, you should never write the partition table to a disk that 
already has a mounted (attached) filesystem; doing so risks damage to the filesystem. 


Note The ioctl operation that causes the kernel to re-read the partition table can fail on rare occasions. 
If such a failure occurs, you need to reboot the system to get the kernel to see the changes. 


After writing the partition table, a new list of partition tables should appear on the console. If you don't see 
this list, run dmesg to see the kernel messages and look at the end of the output. For the fdisk session 
earlier in this section, here is what you would see: 


hdc: hdcl 


Thehda indicates the disk that you repartitioned. The new partition list appears after the colon. For 
example, if you create three partitions on the disk, the output might appear as hdc: hdcl hdc2 hdc3. 


Note Looking at the fdisk session in this section, you may be wondering what the difference between 
a primary and an extended partition is. The standard PC disk-partitioning scheme originally only 
allowed a maximum of four partitions, 1-4. These are the primary partitions. If you want more 
partitions, fdisk can designate one of these primary partitions as an extended partition, 
allowing you to place subpartitions in the extended partition. Each of these subpartitions is called 
alogical partition. 


After partitioning a disk, you're not quite ready to attach it to your system because you must put a 
filesystem on your partition(s) first. 


2.4Filesystems 


Afilesystem is a database of files and directories that you can attach to a Unix system atthe root (/) or 
some other directory (like /usr) in acurrently attached filesystem. At one time, filesystems resided on 
disks and other physical media used exclusively for data storage. However, the tree-like directory structure 
and I/O interface of filesystems is quite versatile, so filesystems now perform a variety of tasks. 


2.4.1Filesystem Types 


Linux supports an extraordinarily large number of filesystems, including native designs optimized for 
Linux, foreign types such as the Windows FAT family, universal filesystems like ISO9660, and others. The 
following list includes the most common types of filesystems for data storage; the type names as 
recognized by Linux are in parentheses next to the boldfaced filesystem names. 


=m TheSecond Extended filesystem (ext2) is native to Linux. It is fairly quick, and it defragments itself. 
Nearly every Linux system uses ext2 or its newer, journaled version, exts. 


= Third Extended filesystems (ext3) are ext2 filesystems augmented with journal support. This can 
make recovery from an abrupt system reboot or failure quicker and less painful. 


=» ISO9660 (iso9660) is a CD-ROM standard. Most CD-ROMs use some variety of ISO9660 extension; 
Linux supports them. 


= FAT filesystems (msdos, vfat, umsdos) pertain to Microsoft systems. The simple msdos type supports 
the very primitive monocase variety in MS-DOS systems. For Windows filesystems, use vfat. The 
umsdos filesystem is peculiar to Linux; it supports Unix features such as symbolic links on top of an 
MS-DOS filesystem. It is also not very common. 


=» TheReiser filesystem (reiserfs) is relatively new. It supports a journal and is optimized for fairly small 
files, a condition that often occurs in Unix systems. 


2.4.2Creating a Filesystem 


You cannot mount and store files on a partition that does not contain a filesystem. The partitioning process 
described in Section 2.3.4 does not create any filesystems; you must place the filesystems on the 
partitions in a separate step. To create a Second Extended (ext2) filesystem, use the mke2fs program on 
the target device, as in this example for /dev/hdc3: 


mke2fs /dev/hdc3 


Themke2fs program automatically determines the number of blocks in a device and sets some 
reasonable defaults. Unless you really know what you're doing and feel like reading the mke2fs(8) manual 
page in detail, you shouldn't change these. 


When you create a filesystem, you initialize its database, including the superblock and the inode tables. 
The superblock is at the top level of the database, and it's so important that mke2fs creates a number of 
backups in case the original is destroyed. You may wish to record a few of the superblock backup numbers 
whenmke2fs runs, in case you need to recover it later in the event of a disk failure (see Section 2.4.8). 


Warning Filesystem creation is a rare task that you should only need to perform after adding a new 
disk or repartitioning an old disk. You should create a filesystem just once for each new 
partition that has no preexisting data (or data that you want to remove). Creating a new 
filesystem on top of an existing filesystem will effectively destroy the old data. 


Creating ext3 Filesystems 
The only substantial difference between ext2 and ext3 filesystems is that ext3 filesystems have a journal 


file containing changes not yet written to the regular filesystem database. To create an ext3 filesystem, use 
the-j option to mke2fs: 


mke2fs -j /dev/ disk_device 


Don't worry if you forget the -j option when creating a filesystem. You can add a journal file to an existing 
filesystem with the utility. Here's an example: 


tune2fs -j /dev/hdal 


When upgrading a filesystem to ext3, don't forget to change the ext2 to ext 3inthe /etc/fstab file. 


2.4.3Mounting a Filesystem 


On Unix, the process of attaching a filesystem is called mounting. When the system boots, the kernel 
reads some configuration data and mounts / based on that data. To mount a filesystem, you must know 
the following: 


= The filesystem's device (such as a disk partition; where the actual filesystem data resides). 


m The filesystem type, or design. Operating system developers use different types to adapt to their 
particular system for backward compatibility or for other reasons that aren't necessarily that good. For 
example, the ext2-/ext3-based filesystems common on Linux are quite different than the FAT -based 
types found on many Windows machines. 


=» Themount point; that is, the place inthe current system's directory hierarchy where the filesystem will 
be attached. The mount point is always anormal directory. For instance, Linux uses /cdromasa 
mount point for CD-ROM devices. The mount point need not be directly below /; it may be anywhere 
on the system. 


When mounting a filesystem, the common terminology is "mount a device on a mount point." To learn the 
current filesystem status of your system, run mount. The output looks like this: 


/dev/hdal on / type ext2 (rw, errors=remount-—ro) 
proc on /proc type proc (rw) 

/dev/hda3 on /usr type ext2 (rw) 

tmpfs on /dev/shm type tmpfs (rw) 

none on /proc/bus/usb type usbdevfs (rw) 


Each line corresponds to one currently mounted filesystem, with items in this order: 


=» The device, such as /dev/hda3. Notice that some of these aren't real devices (proc, for example); 
these are stand-ins for real device names, because these special-purpose filesystems do not need 
devices. 


a The word on. 

=» The mount point. 

=» [he word type. 

= The filesystem type, usually in the form of a short identifier. 

=» Mount options (in parentheses) — see Section 2.4.5 for more details. 


To mount a filesystem, use the mount command as follows with the filesystem type, device, and desired 
mount point: 


mount -t type device mountpoint 


For example, to mount the Second Extended filesystem /dev/hdb3 on /home/extra, use this 
command: 


mount -t ext2 /dev/hdb3 /home/extra 


To unmount (detach) a filesystem, use the umount command: 


umountmountpoint 


SeeSection 2.4.6 for a few more long options. 


2.4.4Filesystem Buffering 


Linux, like other versions of Unix, buffers (caches) all requested changes to filesystems in memory before 
actually writing the changes to the disk. This cache system is transparent to the user and improves 
performance because the kernel can perform a large collection of file writes at once instead of performing 
the changes on demand. 


When you unmount a filesystem with umount, the kernel automatically synchronizes with the disk. At any 
other time, you can force the kernel to write the changes in its buffer to the disk by running the sync 
command. If (for whatever reason) you can't unmount a filesystem before you turn off the system, make 
sure that you run sync first. 


2.4.5Filesystem Mount Options 


There are many ways to change the mount command behavior. This is often necessary with removable 
media or when performing system maintenance. 


The total number of mount options is staggering. The very extensive mount(8) manual page is a good 
reference, but it's hard to know where to start and what you can safely ignore. 


Options fall into two rough categories: general options and filesystem-specific options. General options 
include—t for specifying the filesystem type, which was mentioned earlier. By contrast, a filesystem- 
specific option pertains only to certain filesystem types. To activate a filesystem option, use the -o switch 
followed by the option. For example, -o norock turns off Rock Ridge extensions on an ISO9660 
filesystem, but it has no meaning for any other kind of filesystem. 


Short Options 


The most important general options are the following: 
-r The -r option mounts the filesystem in read-only mode. This has a number of uses, from write 


protection to bootstrapping. You don't need to specify this option when accessing a read-only device 
such as a CD-ROM; the system will do it for you (and will also tell you about the read-only status). 


-n The -n option ensures that mount does not try to update the system mount database, 
/etc/mtab. The mount operation fails when it cannot write to this file. This is important at boot time, 
because the root partition (and therefore, the system mount database) are read-only at first. You will 
also find this option handy if you are trying to fix asystem problem in single-user mode (see Section 
3.2.4), because the system mount database may not be available at the time. 


-t The -ttype option specifies the filesystem type. 


Long Options 


Short options like -r are too limited for the ever-increasing number of mount options; there are too few 
letters in the alphabet to accommodate all possible options. Short options are also troublesome because it 
is difficult to determine an option's meaning based on a single letter. Many general options and all 
filesystem-specific options use a longer, more flexible option format. 


To use long options with mount on the command line, start with -o and supply some keywords. Here is a 
complete example with the long options in boldface: 


mount -t vfat /dev/hdal /dos -o ro, conv=auto 


There are two long options here, ro and conv=auto. The ro option specifies read-only mode, and it is the 
same as the -r short option. The conv=auto option is a filesystem option telling the kernel to 
automatically convert certain text files from the DOS newline format to the Unix style (which will be 
explained shortly). 


The most useful long options are the following: 


exec,noexec Enables or disables execution of programs on the filesystem. 
suid,nosuid Enables or disables setuid programs (see Section 1.17). 
ro,rw Mounts the filesystem as read-only or read-write. 


remount Reattaches a currently mounted filesystem at the same mount point. The only real reason to 
do this is to change mount options, and the most frequent case is making a read-only filesystem 
writable. An example of why you might use this is when the system leaves the root in read-only mode 
during crash recovery. The following command remounts the root in read-write mode (you need the - 
n option because the mount command cannot write to the system mount database when the root is 
read-only): 


mount -n -o remount / 


The preceding command assumes that the correct device listing for / isin /etc/fstab (explained in 
thenext section). If itis not, you must specify the device. 


norock,nojoliet (|SO9660 filesystem) Disables Rock Ridge (Unix) or Joliet (Microsoft) extensions. 
Be warned that plain, raw ISO9660 is really ugly. 


conv=rule (FAT-based filesystems) Converts the newline characters in files based on rule, which 
canbe binary,text, orf auto. The default is binary, which disables any character translation. To 
treat all files as text, use text. The auto setting converts files based on their extension. For 
example, a . jpg file gets no special treatment, but a .t xt file does. Be careful with this option, 
because it can damage files. You may want to use it in read-only mode. 


2.4.6The /etc/fstab Filesystem Table 


To mount filesystems at boot time and take the drudgery out of the mount command, Linux systems keep 
a permanent list of filesystems and options in /etc/fstab. Thisis a plain text file in a very simple 
format, as this example shows: 


/dev/hdal “/ ext2 defaults, errors=remount-ro ~I 
/dev/hda2 none swap SW 0 0 
/dev/hnda3 /usr ext2 defaults 0 2 
proc J/OrSoe proc defaults 0 0 
/dev/hdc /cdrom iso9660 ro,user,nosuid,noauto 0 0 


Each line corresponds to one filesystem, broken into six fields: 


The device. Notice that the /proc entry has a stand-in device. 
The mount point. 


The filesystem type. You may not recognize swap, for /dev/hda2. This is a swap partition (see 
Section 2.5). 


Options. 


Backup information for the dump command; dump does not see common use, but you should always 
specify this field with a 0. 


The filesystem integrity test order (see the fsck command in Section 2.4.8). To ensure that fsck 
always runs on the root first, you should always set this to 1 for the root filesystem and 2 for any other 
filesystems on a hard disk. Use 0 to disable the bootup check for everything else, including CD-ROM 
drives, swap, andthe /proc filesystem. 


When using mount, you can take some shortcuts if the filesystem you want to work with is in 
/etc/fstab. For the example fstab above, to mount a CD-ROM, you need only run 


mount /cdrom 


You can also try to mount all entries in /etc/fstab that do not contain the noauto option at once, with 


this command: 


mount -a 


You may have noticed some new options in the preceding fstab listing, namely defaults,errors, 
noauto, and user. These aren't covered in Section 2.4.5 because they don't make any sense outside of 
the/etc/fstab file. The meanings are as follows: 


defaults This uses the mount defaults — read-write mode, enable device files, executables, the 


setuid bit, and so on. You should use this when you don't want to give the filesystem any special 
options, but you do want to fill all fields in /etc/ fstab. 


errors Ihis ext2-specific parameter sets the system behavior if there is trouble mounting a 
filesystem. The default is normally errors=continue, meaning that the kernel should return an 


error code and keep running. To get the kernel to try again in read-only mode, use 
errors=remount-ro. The errors=panic setting tells the kernel (and your system) to halt when 


there is a problem. 


noauto [his option tells amount -a command to ignore the entry. Use this to prevent a boot-time 
mount of a removable-media device, such as a CD-ROM or floppy drive. 


user [his option allows normal users to run mount on this entry. This can be handy for enabling 


access to CD-ROM drives. Because users can put a setuid-root file on removable media with another 
system, this option also sets nosuid,noexec, and nodev (to bar special device files). The fstab 
example in this section explicitly sets nosuid. 


2.4./Filesystem Capacity 


To view the size and utilization of your currently mounted filesystems, use the df command. The output 
looks like this: 


Filesystem 1024-blocks Used Available Capacity Mounted on 
/dev/hdal LOTL92¢ 71400 889124 7% i 
/dev/hda3 17710044 9485296 1325106 56% /usr 


The listing has the following fields: 


Filesystem The filesystem device 

1024-blocks The total capacity of the filesystem in blocks of 1024 bytes 
Used The number of occupied blocks 

Available The number of free blocks 

Capacity The percentage of blocks in use 


Mounted on The mount point 


It is relatively easy to see that the two filesystems here are roughly 1GB and 17.5GB in size. However, the 
Capacity numbers may look a little strange because 71400 + 889124 does not equal 1011928, and 
9485296 does not constitute 56 percent of 17710044. In both cases, 5 percent of the total capacity is 
unaccounted for. Nevertheless, the space is there. These hidden blocks are called the reserved blocks, 
and only the superuser may use the space if the rest of the partition fills up. This keeps system servers 
from immediately failing when they run out of disk space. 


If your disk fills up and you need to know where all of those space-hogging, illegal MP3s are, use the du 


command. With no arguments, du prints the disk usage of every directory in the directory hierarchy, 
starting at the current working directory. (That's kind of a mouthful, so just run cd /; dutogetthe idea. 
Press CONTROL-C when you get bored.) The du -s command turns on summary mode to print only the 
grand total. If you want to evaluate a particular directory, change to that directory and run du -s *. 


Note 1024-byte blocks in df and du output is not the POSIX standard. Some systems insist on 
displaying the numbers in 512-byte blocks. To get around this, use the -k option (both utilities 
support this). The df program also supports the -m option to list capacities in one-megabyte 
blocks. 


The following pipeline is a handy way to create a searchable output file (du_out) and see the results on 
the terminal at the same time. 


du | tee du_out 


2.4.8Checking and Repairing Filesystems 


The optimizations that Unix filesystems offer are made possible by a sophisticated database-like 
mechanism. For filesystems to work seamlessly, the kernel has to trust that there are no errors in a 
mounted filesystem. Otherwise, serious errors such as data loss and system crashes can happen. 


The most frequent cause of a filesystem error is shutting down the system in a rude way (for example, with 
the power switch on the computer). The system's filesystem cache in memory may not match the data on 
the disk, and the system also may be in the process of altering the filesystem when you decide to give the 
computer a kick. Even though a new generation of filesystems supports journals to make filesystem 
corruption far less common, you should always shut the system down properly (see Section 3.1.5). 
Furthermore, filesystem checks are still necessary every now and then as sanity checks. 


You need to remember one command name to check a filesystem: fsck. However, there is a different 
version of this tool for each filesystem type that Linux supports. The information presented here is specific 
to second and third extended (ext2/ext3) filesystems and the e2fsck utility. You generally don't need to 
typee2 fsck, though, unless fsck can't figure out the filesystem type, or you're looking for the e2fsck 
manual page. 


To run fsck in interactive manual mode, use the device or the mount point (in /etc/fstab) as the 
argument. For example: 


fsck /dev/hddl 


Warning Never use fsck on a mounted filesystem. The kernel may alter the disk data as you run the 


check, causing mismatches that can crash your system and corrupt files. There is only one 
exception. If you mount the root as read-only in single user mode, you may use fsck onthe 


root filesystem. 


In manual mode, fsck prints verbose status reports on its passes, which should look something like this 
when there are no problems: 


Pass 1: Checking inodes, blocks, and sizes 
Pass 2: Checking directory structure 

Pass 3: Checking directory connectivity 
Pass 4: Checking reference counts 


Pass 5: Checking group summary information 
/dev/hdd1l: 11/1976 files (0.0% non-contiguous), 265/7891 blocks 


Iffsck finds a problem in manual mode, it stops and asks you a question relevant to fixing the problem. 
These questions deal with the internal structure of the filesystem, such as reconnecting loose inodes and 
clearing blocks. The reconnection business means that fsck found a file that doesn't appear to have a 
name; reconnecting places the file inthe lost+found directory filesystem as a number. You need to 
guess the name based on the content of the file. 


In general, it's pointless to sit through the fsck process if you just made the mistake of an impolite 


shutdown.e2 fsck has a -p option to automatically fix silly problems without asking you, aborting if there 
IS a serious error. This is so common that Linux distributions run some variant of fsck -p at boot time 
(fsck -a is also common). 


However, if you suspect that there is some major disaster, such as a hardware failure or device 
misconfiguration, you need to decide on a course of action, because fsck can really mess up a filesystem 
with larger problems. A telltale sign of a serious problem is a /ot of questions in manual mode. 


If you think that something really bad happened, try running fsck -n to check over the filesystem without 
modifying anything. If there's some sort of problem with the device configuration (an incorrect number of 
blocks in the partition table, loose cables, whatever) that you think you can fix, then fix it before running 
fsck for real. You're likely to lose a lot of data otherwise. 


If you suspect that only the superblock, a key filesystem database component, is corrupt (for example, 
someone wrote to the beginning of the disk partition), you might be able to recover the filesystem with one 
of the superblock backups that mke2fs creates. Use fsck -b num to replace the corrupted superblock 
with an alternate at block num. 


You may not know where to find a backup superblock, because you didn't write the numbers down when 
mke2fs ran. If the filesystem was created with the default values, you can try mke2fs -n onthe device 
to view alist of superblock backup numbers without destroying your data (again, make deadsure that 
you're using —n, because you'll really tear up the filesystem otherwise). 


If the device still appears to function properly except for a few small parts, you can run fsck -c before a 
manualfsck to search for bad blocks. Such a failure is somewhat rare. 


Checking ext3 Filesystems 


You normally do not need to check ext3 filesystems because the journal ensures data integrity. However, 
you may wish to mount an ext3 filesystem in ext2 mode. The kernel will not mount an ext3 filesystem that 
contains a non-empty journal (if you don't shut your system down cleanly, you can expect that the journal 
contains some data). To flush the journal in an ext3 filesystem to the regular filesystem database, run 
e2fsck as follows: 


e2fsck -fy /dev/disk_device 


The Worst Case 
Disk problems that are worse in severity leave you with few choices: 


=» You can try to pull the entire filesystem from the disk with dd and transfer it to a partition on another 
disk that's the same size. 


=» You could try to patch up the filesystem as well as you can, mount it in read-only mode, and salvage 
what you can. 


In both cases, you still need to repair the filesystem before you mount it (unless you feel like picking 
through the raw data by hand). To answer y to all of the fsck questions, use fsck -y, but do this as a 
last resort. 


Note There is an advanced utility called debugfs for users with in-depth knowledge of filesystems, or 
for those who feel like experimenting on a filesystem that isn't important. 


If you're really desperate, such as in the event of a catastrophic disk failure without backups, there isn'ta 
lot you can do other than try to get a professional service to "scrape the platters." 


2.4.9Special-Purpose Filesystems 


Not all filesystems represent storage on physical media. Most versions of Unix have filesystems that serve 
as system interfaces. This idea goes back along way; the /dev mechanism is an early model of using files 
for I/O interfaces. The /proc idea came from the eighth edition of research Unix [Killian]. Things really 
got rolling when the people at Bell Labs (including many of the original Unix designers) created Plan 9 
[Bell Labs], a research operating system that took filesystem abstraction to a whole new level. 


The special filesystem types in common use on Linux include the following: 
proc, mounted on /proc. The name "proc" is actually an abbreviation of "process." Each numbered 


directory inside /proc Is actually the process ID of a current process on the system, and the files in 
those directories represent various aspects of the processes. /proc/self represents the current 
process. The Linux proc filesystem includes a great deal of additional kernel and hardware 
information, such as /proc/cpuinfo. Purists shudder and say that this additional information does 
not belong in /proc, but rather in /dev or some other directory, but it's probably too late to change it 
in Linux now. 


usbdevfs, mounted on /proc/bus/usb. Programs that interact with the USB interface and its 
devices often need the files here. The files in a usbdevfs filesystem provide interesting information on 
the bus status. 


tmpfs, mounted on /dev/shm. You can employ your physical memory and swap space as temporary 
storage with tmpfs. You can mount tmpfs wherever you like, using the size and nr_blocks long 
options to control the maximum size. However, you must be careful not to pour things into a tmpfs, 
because your system will eventually run out of memory, and programs will start to crash. For years, 
Sun systems used a version of tmpfs for /tmp, and this is a frequent problem on long-running 
systems. 


2.5Swap and Virtual Memory 


If you run out of real memory, Linux has a virtual memory system that automatically moves memory 
pages (chunks) to and from a hard disk. This is called swapping, because the pages of idle programs are 
swapped to the disk in exchange for active pages residing on the disk. The disk area used to store memory 
pages is called swap space. 


Thefree command's output includes the current swap usage in kilobytes as follows: 


total used free 


Swap: 514072 189804 324268 


2.5.1Using a Disk Partition as Swap Space 


To use an entire disk partition as swap, follow these steps: 
1. Make certain that the partition is empty. 


2. Runmkswapdev, where dev is the partition's device. This command puts a swap signature on the 
partition. 


3. Executeswapondev to register the space with the kernel. 


After creating a swap partition, you can put a new swap entry in your /etc/fstab to make the system use 
the swap space as soon as the machine boots. This entry is from the fstab example in Section 2.4.6: 


/dev/hda2 none swap SW 0 0 


2.5.2Using a File as Swap Space 


You can also use a regular file as swap space. It's not quite as fast as a swap partition, but if you're ina 
pinch where you would be forced to repartition the disk to use a swap partition, using a file as swap space 
works fine. 


Use these commands to create an empty file, initialize it as swap, and add it to the swap pool: 


dd if=/dev/zero of=swap_file bs=1024k count=num_mb 
mkswapswap_file 
Swaponswap_file 


Here, swap_file isthe name of the new swap file, and num_mb is the desired size, in megabytes. 


To remove a swap partition or file from the kernel's active pool, use the swapoff command. 


2.5.3How Much Swap do You Need? 


At one time, the Unix conventional wisdom said that you should always reserve at least twice as much 
swap as you have real memory. The enormous disk and memory capacities now available cloud the issue. 
On one hand, disk space is so plentiful that double the memory size may seem inadequate, given the 
amount of disk space available. On the other hand, you may never even dip into your swap space because 
you have so much real memory. 


Some things never change, though. Reserve two to five times as much disk space as you have real 
memory for swap. It doesn't make sense to go any lower, because you may actually risk running out of 
memory. If you go higher and actually intend to use all of this swap space, you will likely suffer serious 
performance problems because the system will spend all of its time swapping (a condition known as 
thrashing). 


Chapter 3: How Linux Boots 


Overview 


You now know the physical structure of a Linux system, what the kernel is, and how to work with 
processes. This chapter teaches you how the system starts (boots) — that is, how the kernel gets into 
memory and how the regular system processes get started. 


As it turns out, there isn't much to the boot process: 
1. A boot loader finds the kernel image on the disk, loads it into memory, and starts it. 


. The kernel initializes the devices and its drivers. 


. The kernel mounts the root filesystem. 


2 

3 

4. The kernel starts a program called init. 

5. init sets the rest of the processes in motion. 
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. The last processes that init starts as part of the boot sequence allow you to log in. 


Identifying each stage of the boot process is invaluable in fixing boot problems and understanding the 
system as a whole. To start, zero in on the boot loader, which is the initial screen or prompt you get after 
the computer does its power-on self-test, asking which operating system to run. After you make a choice, 
the boot loader runs the Linux kernel, handing control of the system to the kernel. 


There is a detailed discussion of the kernel elsewhere in this book (Section 2.2 explains the role of the 
kernel, and Chapter 10 tells you how to build one yourself), but this chapter covers the kernel initialization 
stage, the stage when the kernel prints a bunch of messages about the hardware present on the system. 
The kernel starts init just after it displays a message proclaiming that the kernel has mounted the root 
filesystem: 


VFS: Mounted root (ext2 filesystem) readonly. 


Soon after, you will see a message about init starting, followed by system service startup messages, 
and finally you get a login prompt of some sort. 


Note On Red Hat Linux, the init note is especially obvious, because it "welcomes" you to "Red Hat 
Linux." All messages thereafter show success or failure in brackets at the right-hand side of the 
screen. 


Most of this chapter deals with init, because it is the part of the boot sequence where you have the most 
control.Section 3.2 deals with the boot loaders. 


3.1init 


There is nothing special about init. It is a program just like any other on the Linux system, and you'll find 
itin /sbin along with other system binaries. The main purpose of init is to start and stop other programs 
in a particular sequence. All you have to know is how this sequence works. 


There are a few different variations, but most Linux distributions use the System V style discussed here. 
Some distributions use a simpler version that resembles the BSD init, but you are unlikely to encounter 
this. 


3.1.1Runlevels 


At any given time on a Linux system, a certain base set of processes are running. This state of the 
machine is called its runlevel, and it is denoted with a number from 0 through 6. The system spends most 
of its time ina single runlevel. However, when you shut the machine down, init switches to a different 
runlevel in order to terminate the system services in an orderly fashion and to tell the kernel to stop. Yet 
another runlevel is for single-user mode, discussed in Section 3.2.4. 


The easiest way to get a handle on runlevels is to examine the init configuration file, /etc/inittab. 
Look for a line like the following: 


1d:5:initdefault: 


This line means that the default runlevel on the system is 5. All lines in the inittab file take this form, 
with four fields separated by colons occuring in the following order: 


m A unique identifier (a short string, such as id in the preceding example) 
= The applicable runlevel number(s) 


a» Theaction that init should take (in the preceding example, the action is to set the default runlevel to 
5) 


» A command to execute (optional) 


There is no command to execute in the preceding initdefault example because a command doesn't 
make sense in the context of setting the default runlevel. Look a little further down in inittab, until you 
see a line like this: 


L525 mait i/e oro dro L 


This particular line is important because it triggers most of the system configuration and services through 
therc*.dand init.d directories (see Section 3.1.2). You can see that init is set to execute a 
command called /etc/rce.d/re 5 when in runlevel 5. The wait action tells when and how init runs 
the command: run rc 5 once when entering runlevel 5, and then wait for this command to finish before 
doing anything else. 


There are several different actions in addition to initdefault and wait, especially pertaining to power 
management, and the inittab(5) manual page tells you all about them. The ones that you're most likely to 
encounter are explained in the following sections. 


respawn 


Therespawn action causes init to run the command that follows, and if the command finishes 
executing, to run it again. You're likely to see something similar to this line in your inittab file: 


1:2345:respawn:/sbin/mingetty ttyl 


Thegetty programs provide login prompts. The preceding line is for the first virtual console 
(/dev/tty1), the one you see when you press ALT-F1 or CONTROL-ALT-F1 (see Section 3.3). The 
respawn action brings the login prompt back after you log out. 


ctrlaltdel 


Thectrlaltdel action controls what the system does when you press CONTROL-ALT-DELETE ona 
virtual console. On most systems, this is some sort of reboot command, using the shut down command, 
which is covered in Section 3.1.5. 


sysinit 


Thesysinit action is the very first thing that init should run when it starts up, before entering any 
runlevels. 


3.1.2How Processes in Runlevels Start 


You are now ready to learn how init starts the system services, just before it lets you log in. Recall this 
inittab line from earlier: 


1525 2waitls/ ecc/ re.d7rc. 5 


This small line triggers many other programs. rc stands for run commands, and you will hear people refer 
to the commands as scripts, programs, or services. So, where are these commands, anyway ? 


For runlevel 5, in this example, the commands are probably either in /etc/rce.d/rc5.dor /etc/rc5.d. 
Runlevel 1 uses rcl1.d, runlevel 2 uses rc2.d, and so on. You might find the following items in the 
rc5.d directory: 


SlLOSvskilogd Ss20ppp S99gpm 
S12kerneld S25netstd_nfs Sone L pa 
SLSneEStd init ss0netstd mise SO9rmnologin 
S1i8netbase S45pcmcia So ossid 
Sz0ace. 569atd 

S20Llogoutad sS8&89cron 


Therc 5 command starts programs in this runlevel directory by running the following commands: 


Sl0sysklogd start 
Sizkerneld start 
Sidnetstd_ init start 
Sl8netbase start 


S99sshd start 


Notice the start argument in each command. The s in a command name means that the command 
should run in start mode, and the number (00 through 99) determines where in the sequence rc Starts the 
command. 


Therc*.d commands are usually shell scripts that start programs in /sbin or /usr/sbin. Normally, 
you can figure out what one of the commands actually does by looking at the script with less or another 
pager program. 


You can start one of these services by hand. For example, if you want to start the httpd Web server 
program manually, run S99httpd start. Similarly, if you ever need to kill one of the services when the 
machine is on, you can run the command in the rc* .a directory with the stop argument (S99httpd 
stop, for instance). 


Somerc* .d directories contain commands that start with with K (for "kill," or stop mode). In this case, rc 
runs the command with the stop argument instead of start. You are most likely to encounter K 
commands in runlevels that shut the system down. 


3.1.3Adding and Removing Services 


If you want to add, delete, or modify services in the rc* .d directories, you need to take a closer look at 


the files inside. A long listing reveals a structure like this: 


lrwxrwxrwx . . . S1l0sysklogd -> ../init.d/sysklogd 
lrwxrwxrwx . . . Sl2kerneld -> ../init.d/kerneld 
lewerwerwx « « « Slbnetsed init => a4/1init:-d/netstd init 
lrwxrwxrwx . . . Sl8netbase -> ../init.d/netbase 


The commands in an rc*.d directory are actually symbolic links to files in an init .d directory, usually in 
/etc or /etc/rc.d. Linux distributions contain these links so that they can use the same startup scripts 
for all runlevels. This convention is by no means a requirement, but it often makes organization a little 
easier. 


To prevent one of the commands in the init.d directory from running in a particular runlevel, you might 
think of removing the symbolic link in the appropriate rc* .d directory. This does work, but if you make a 
mistake and ever need to put the link back in place, you might have trouble remembering the exact name 
of the link. Therefore, you shouldn't remove links in the rc* .d directories, but rather, add an underscore 

(_) to the beginning of the link name like this: 


my S99httpd —_So9httpd 


At boot time, rc ignores _S99httpd because it doesn't start with s or K. Furthermore, the original name is 
still obvious, and you have quick access to the command if you're in a pinch and need to start it by hand. 


To add a service, you must create a script like the others in the init.d directory and then make a 
symbolic link in the correct rc* .d directory. You may need to read Chapter 7 first if you aren't familiar 
with shell scripts. However, the easiest way to write a script is to examine the scripts already in init.d, 
make a copy of one that you understand, and modify the copy. 


When adding a service, make sure that you choose an appropriate place in the boot sequence to start the 
service. If the service starts too soon, it may not work, due to a dependency on some other service. For 
non-essential services, most systems administrators prefer numbers in the 90s, after most of the services 
that came with the system. 


Linux distributions usually come with a command to enable and disable services in the rc* .d directories. 
For example, in Debian, the command is update-rc.d, and in Red Hat Linux, the command is 
chkconfig. Graphical user interfaces are also available. Using these programs helps keep the startup 
directories consistent and helps with upgrades. 


HINT One of the most common Linux installation problems is an improperly configured XFree86 
server that flicks on and off, making the system unusable on console. To stop this behavior, boot 
into single-user mode and alter your runlevel or runlevel services. Look for something containing 
xdm,gdm Or kdm in your rc*.d directories, or your /etc/inittab. 


3.1.4Controlling init 


Occasionally, you need to give init a little kick to tell it to switch runlevels, to re-read the inittab file, 
or just to shut down the system. Because init is always the first process on a system, its process ID is 
always 1. 


You can control init with telinit. For example, if you want to switch to runlevel 3, use this command: 


telinit 3 


When switching runlevels, init tries to kill off any processes that aren't in the inittab file for the new 
runlevel. Therefore, you should be careful about changing runlevels. 


When you need to add or remove respawning jobs or make any other change to the inittab file, you 
must tell init about the change and cause it to re-read the file. Some people use kill -HUP 1 to tell 
init todo this. This traditional method works on most versions of Unix, as long as you type it correctly. 
However, you can also run this telinit command: 


telinit qg 


You can also use telinit s to switch to single-user mode (see Section 3.2.4). 


3.1.5Shutting Down 


init also controls how the system shuts down and reboots. The proper way to shut down a Linux machine 
is to use the shutdown command. 


There are two basic ways to use shutdown. If you ha/tthe system, it shuts the machine down and keeps it 
down. To make the machine halt immediately, use this command: 


shutdown -h now 


On most modern machines with reasonably recent versions of Linux, a halt cuts the power to the machine. 
You can also reboot the machine. For a reboot, use -r instead of —h. 


The shutdown process takes several seconds. You should never reset or power off a machine during this 
stage. In the preceding example, now is the time to shut down. This argument is mandatory, but there are 
many ways of specifying it. If you want the machine to go down sometime in the future, one way is to use 
+n, where n is the number of minutes shutdown should wait before doing its work. For other options, look 
at the shutdown(8) manual page. 


To make the system reboot in ten minutes, run this command: 


shutdown -r +10 


On Linux, shutdown notifies anyone logged on that the machine is going down, but it does little real work. 
If you specify a time other than now,shutdown creates a file called /etc/nologin. When this file is 
present, the system prohibits logins by anyone except the superuser. 


When system shutdown time finally arrives, shutdown tells init to switch to runlevel 0 for a halt and 
runlevel 6 for a reboot. When init enters runlevel O or 6, all of the following takes place, which you can 
verify by looking at the scripts inside rcO.d and rc6.d: 


1. init kills every process that it can (as it would when switching to any other runlevel). 


2. The initial rcO.d/rc6é.d commands run, locking system files into place and making other 
preparations for shutdown. 


3. The next rc0.d/rc6.d commands unmount all filesystems other than the root. 
4. Furtherrc0.d/rc6.d commands remount the root filesystem read-only. 


5. Still more rcO.d/rc6é.d commands write all buffered data out to the filesystem with the sync 
program (see Section 2.4.4). 


6. The final rc0.d/rc6.d commands tell the kernel to reboot or stop with the reboot,halt, or 
poweroff program. 


Thereboot and halt programs behave differently for each runlevel, potentially causing confusion. By 
default, these programs call shut down with the -r or -h options. but if the system is already at the halt or 
reboot runlevel, the programs tell the kernel to shut itself off immediately. If you really want to shut your 
machine down in a hurry (disregarding any possible damage from a disorderly shutdown), use the -£f 
option. 


3.2Boot Loaders 


Before the kernel runs init, a boot loader starts the kernel. On occasion, you need to tell the boot loader 
to load different kernels or operating systems, and to start in different modes. This is especially important 
when trying to fix a system that has a problem that prevents a full boot. To fix such a problem, you may 
need single-user mode or an alternate kernel. 


The boot loader loads a kernel image into memory and hands control of the CPU to the new image, 
possibly supplying it with some parameters. These parameters are simple text strings like -s for booting in 
single-user mode and root=partition for using partition as the root filesystem instead of the 
default. You can specify a runlevel number as a parameter to make the system boot into a runlevel other 
than the default. 


To type in a kernel name and parameters, however, you first need to know how to get to a boot prompt. 
Unfortunately, there are several different boot loaders out there, and because you can control a boot 
loader's behavior, Linux distributions customize to their hearts' content. 


The next sections tell you how to get to a boot prompt in order to enter a kernel name and parameters. If 
you need to know how to install a boot loader or change Its configuration, see Section 10.5. 


3.2.1LILO 


LILO (Linux Loader) has been around for almost as long as the Linux kernel. The LILO boot prompt usually 
ends with boot :. If your system boots with LILO, it's likely that you get a fancy screen of graphics at boot 
time, because this is the default for many distributions, including Red Hat Linux. If you see a screen like 
this, look for a part that reads "Press Control-x for text mode." If you see that message, tyoe CONTROL-X 
to get to the boot prompt. 


If the system defaults to text mode, look at the prompt as soon as it appears. If the prompt says LILO with 
nothing else, press the SHIFT key to get the rest of the boot prompt to appear. On the other hand, if you 
get a boot prompt immediately, you need to watch out, because the system likely will boot if you don't type 
anything in a certain amount of time. 


Once you're at a LILO boot prompt, press the TAB key to show a list of kernel and operating system 


options. The default kernel name is probably something like 1inux. To boot this kernel with no options, 
enterlinux. To use options, specify them after the kernel name: 


linuxoptionl option2 


For example, to boot the system in single-user mode, type this: 


linux -s 


Or, to boot 1inux in single-user mode with the root filesystem as /dev/hda3 instead of your normal 
default, type this: 


linux root=/dev/hda3 -s 


3.2.2GRUB 


GRUB stands for Grand Unified Bootloader, a system that is slowly replacing LILO. GRUB has plenty of 
whiz-bang features, but most important is its ability to navigate filesystems, so you can read files without 
loading a kernel. Wiseguy Solaris and BSD administrators like to say that they have enjoyed this capability 
for some time. 


GRUB has a menu interface that's easy enough to navigate, but if you need to boot from a different kernel, 
change the root partition, or supply extra kernel parameters, you should get into the mini-shell. Press c at 
the menu to get this prompt: 


grub> 


Let's say that you want to boot the kernel at /boot /vmlinuz witha root of /dev/hda3. Furthermore, 
your system is messed up, so you need single-user mode with the -s kernel option. Type the following at 
the GRUB prompt: 


root (Adv; 2) 
kernel /boot/vmlinuz root=/dev/hda3 -s 
boot 


Theroot (hd0,2) line sets GRUB's idea of the current root partition — that is, the filesystem where 
GRUB expects to find the kernel. hado is the first hard drive (that is, the first disk that GRUB finds; for 
example, the Linux device /dev/hda if this is your first hard disk). However, 2 specifies the third partition 
(/dev/hda3) because GRUB partition numbers start at 0. 


The word kernel sets a kernel image and its parameters. /boot /vmlinuz refers to a kernel image file 
on (hd0, 2). Unfortunately, GRUB does not normally pass the information from the preceding root () 
line on to the kernel, so you should always specify root=partition asa kernel parameter. 


Note You can combine the root and kerne1 lines by preceding the kernel image with the GRUB root 
partition. Therefore, the preceding two lines could be written as the single line kernel 
(hdd0,2)/boot/vmlinuz root=/dev/hda3 -s. 


The last line, boot, tells GRUB to load and execute the kernel image. 


Note On certain systems (especially those with SCSI disks and stock kernels), you may need an initial 
RAM disk: 


initrd (boot initrd 


SeeSection 10.5.4 for information on why you may need an initial RAM disk. 


In case you're trying to boot a partition with another boot loader (such as a Windows partition) by hand with 
GRUB, try the following, where partition uses the GRUB device syntax explained earlier (e.g., 
hd(0,1)): 


rootnoverifypartition 
makeactive 
chainloader +1 

boot 


3.2.30ther Boot Loaders 


There are many other ways to boot a kernel, including from DOS via LOADLIN or SYSLINUX, over the 
network, or even directly from the PC BIOS with LinuxBIOS! Most other boot loaders work like LILO, 
although some do not support the TAB key to list options. You may need to pay attention to boot 
diagnostics for this information. However, the way you enter parameters is usually identical to LILO and 
GRUB. 


3.2.4Single-User Mode and Emergency Booting 


When something goes wrong with the system, an administrator's first recourse for booting quickly to a 
usable state is single-user mode. The idea is that the system quickly boots to a root shell instead of going 
through the whole mess of services. On Linux, single-user mode is usually runlevel 1. You may need to 
type the root password to enter single-user mode. 


Common tasks in single-user mode include the following: 
m Checking filesystems after a system crash 


a Fixing problems in critical files, such as /etc/fstab,/etc/passwd, and /etc/inittab 


=» Restoring from backups after a system crash 


Don't expect too many amenities in single-user mode. You may need to set the terminal type (enter 
TERM=1inux) to get full-screen editors to work, and the network may not be available. You can configure 
the network and other systems by hand if necessary, but it's a pain. 


When you finish with single-user mode, you can exit the shell to see if the system starts normally. 
However, it's usually a good idea to reboot the system, because the transition from single-user mode to 
regular multi-user mode is not always perfect. 


If you have a real mess on your hands, and even single-user mode doesn't work, you can try the -b kernel 
parameter for an emergency boot shell instead of any kind of orderly startup. This does not mount the root 
filesystem as read-write or give you much of anything else, so you'll probably need to do some 
remounting, and possibly mount the /proc filesystem before getting anything useful done. However, if 
things are really this broken, you might consider using a rescue CD-ROM instead of this rudimentary 
emergency boot shell mode. You may also be able to get an emergency boot shell by using the 
init=/bin/sh kernel parameter. 


Finally, if you break your kernel or boot loader, you won't be able to get to single-user mode without extra 
help. You can often boot your system with a kernel from a bootable CD-ROM by passing the root 
parameter to the CD-ROM's kernel boot loader. Your system might look somewhat strange without your 
regular kernel, but you should still be able to move files around or perhaps even compile a new kernel to 
get yourself out of the jam. 


3.3 Virtual Consoles 


The final stage of the boot process starts one or more programs that allow you to log in to the system 
console. Linux has two primary display modes: console (text) mode, and an X Window System server 
(graphics mode, usually via a display manager). The kernel boots in console mode, but on most 
distributions the system switches over to graphics mode near when the rc* .d commands complete. 


Linux has several virtual consoles. Each virtual console may run in graphics or text mode. When in text 
mode, you can switch between consoles with an ALT-Function key combination — for example, ALT-F1 
takes you to /dev/tty1, ALT-F2 goes to /dev/tty2, and so on. 


A virtual console used by XFree86 in graphics mode is slightly different. Rather than getting a virtual 
console assignment directly from /etc/inittab, an XFree86 server takes over a free virtual console. 
For example, if you have getty processes running on tty1 and tty2, anew XFree86 server takes over 
tty3. In addition, after XFree86 puts a virtual console into graphics mode, you must normally press a 
CONT ROL-ALT-Function key combination to switch to another virtual console instead of the simpler ALT- 
Function key combination. 


The upshot of all of this is that if you want to see your text console after your system boots, press 
CONTROL-ALT-F1. To get back to the X11 session, press ALT-F2, ALT-F3, and so on, until you get to the 
X session. 


Chapter 4: Essential System Files, Servers, and 
Utilities 
Overview 


When you first look in the /etc directory or do a process listing, you may feel overwhelmed. Fortunately, 
very few of these files are terribly critical to the system's operation. This chapter shows you what is 
important. 


The subject material in this chapter covers the parts of the system that make the infrastructure covered in 
Chapter 2 available to the user-level tools covered in Chapter 1. In particular, you will see the following: 


=» Configuration files that the system libraries access to get server and user information 
a Server programs (sometimes called daemons) that run when the system boots 
=» Configuration utilities that can be used to tweak the server programs and configuration files 


As in the previous chapters, there is virtually no networking material here. The network is separate from 
the elementary parts of the system. In Chapter 5 you'll see where the network fits in. 


4.1System Logging 


Most system programs write their diagnostic output to the syslog service. The syslogd daemon waits for 

messages, and depending on the type of message received, funnels the output to a file, the screen, users, 
some combination of these, or just ignores it. The system logger is one of the most important parts of the 

system. When something goes wrong and you don't know where to start, you should check the system log 
files first. Here is a sample log file message: 


Aug 19 17:59:48 duplex sshd[484]: Server listening on 0.0.0.0 port 22. 


Most Linux distributions use the /var/log directory to store log files, but you can find out for sure by 
looking at syslog.conf, the syslogd configuration file. It contains lines like this: 


kern.* /dev/console 

* info; authpriv.none /var/log/messages 
authpriv.* (Vary Log/ secure, root 
mail.* /var/log/maillog 
Cron.” /var/log/cron 

* ,emerg x 

1LoGal7 «* /var/log/boot.log 


The type of information to be logged is on the left, and the place where it is logged to is on the right. Most 
of the log targets in this example are normal files, but there are some exceptions: /dev/consoleisa 
special device for the system console, root means to send a message to the superuser if that user is 
logged in, and * means to send a message to all users currently on the system. You can also send 
messages to another host with @host. 


Log messages carry a facility and a priority. The facility is a general category of message (Check the 
syslog.conf(5) manual page for a current list of all facilities). Most are fairly obvious from their name 
(mail, for example). The preceding syslog. conf sample catches messages carrying the authpriv, 
mail,cron, and local7 facilities. In line 6, * is a wildcard that catches output related to all facilities. 


In the syslog.conf file, the priority follows the dot (.) after the facility. The priority may be one of debug, 
info,notice,warning,err,crit,alert, Or emerg. This is also the order of the priorities, with 
emerg being the highest. 


Note You can exclude log messages from a facility in syslog. conf by specifying a priority of none, 
as shown on line 2 of the preceding example. 


It is important to understand that when you put a specific priority ina syslog.conf line, syslogd sends 
messages with that priority and all higherpriorities to the destination on that line. Therefore, in the 
preceding example file, the * . info actually catches most syslog messages and puts them into 
/var/log/messages because info isa relatively low priority. You can catch all priorities by using * as 
a wildcard. 


An extra daemon called k1logd traps kernel messages and (usually) sends them to syslogd. You may 
wish to direct kernel messages into a separate file with a line like this in your syslog.conf: 


kéern.* /var/log/kern.log 


If you want to send a log message to the syslog service manually (for example, when writing a script), use 
thelogger command, as in this example: 


logger -p daemon.info something bad just happened 


Very little can go wrong with syslogd. The most common problem is where syslog.conf doesn't catch 
a certain facility or priority. Another problem is that log files have a tendency to fill their disk partitions after 
some time. However, most distributions automatically trim the files in /var/log with automatic 
invocations of logrotate or some other utility. 


The Linux syslogd has a few more features than its counterparts in other Unix variants. The discussion 
here only covers the basics that apply to most Unix systems. If you're really interested in it, or if there's 
some character in your configuration file that you can't figure out, take a look at the syslog.conf(5) manual 
page. 


4.2A Glance at /etc 


Most configuration files on a Linux system are in /etc, and because there are so many packages on a 
Unix system, /etc accumulates files quickly. This makes it hard to see what actually matters to a running 
system. 


In previous chapters, you saw files like inittab and fstab for booting the system. Table 4-1 identifies 
those and other critical configuration files and where they are discussed in the book. 


Table 4-1: Essential Configuration Files 





File | Function | Section 


fstab Filesystems 2.4.6 
group User management 4.3.2 
init.d Boot sequence ES: 
inittab | Boot sequence | 3.1.1 
ld.so.cache Shared libraries 8.1.4 
1d. sO.2conr Shared libraries 8.1.4 


passwd | User management | 4.3 











tered | Boot sequence | 3.1.2 


shadow User management 4.3 








4.3User Management Files 


The user system in Unix allows different people to log in to the same machine without interfering with one 
another. At the lowest level, users are nothing more than numbers (user IDs), but login names help to 
avert confusion and boredom. 


The plain-text file /etc/passwd maps login names to user IDs, and it looks something like this: 


root ss: 020: Super —usEers7 roots 7 bin/sh 
daemon:*:1:1:daemon:/usr/sbin:/bin/sh 

bins *2 292 bins bin: 7>bin/sh 

SYS 2*2325+6ys7/devi7 bin sh 

nobody: *:65534:65534:nobody:/home: /bin/false 
juser:x:3119:1000:J. Random User:/home/juser:/bin/bash 
beazley:x:143:1000:David Beazley: /home/beazley:/bin/bash 


The format is straightforward. Each line represents one user, with seven fields separated by colons: 
=» Theuser (login) name. 


a [he user's encrypted password. On most Linux systems, the password is not actually stored in the 
passwd file, butin the shadow file. shadow is similar to passwd, but normal users do not have read 
permission on shadow. The second field in passwd or shadow Is the encrypted password, and it 
looks like a bunch of garbage, such as d1CVEWiB/oppc (in Unix, passwords are never stored in clear 
text). The rest of the fields in shadow deal with matters such as password expiration dates. 


Anx in the second field in the passwd file indicates that the encrypted password is stored in shadow. 
An* indicates that the user cannot log in, and if the field is blank (that is, appears as : :), no 

password is required to log in. Watch out for blank passwords. You should never have a user without a 
password. 


= Theuser ID (UID), the user's representation to the kernel. It should be unique. You can have two 
entries with the same user ID, but it will confuse you and some of your software. 


= Thegroup ID (GID). It should be one of the numbered entries in the /etc/group file. Groups 
determine file permissions and little else. This group is also called the user's primary group. 


=» The user's real name (often called the GECOS field). Sometimes there are commas in this field, 
denoting room and telephone numbers. 


=» Theuser's home directory. 
=» The user's shell (the program that runs when the user logs in). 


Figure 4-1 identifies the various fields in one of the entries in the preceding example. 


legin mame 
Password 
Hi 
[M 
rHone directory 

Shell 

eo 

juger:x:311 l m User: /home/juser in/bash 


Figure 4-1: An aan in the Seeder fle. 
Note A user and home directory are collectively known as an account. 


There are a few special users in /etc/passwd. The superuser (root) always has UID 0 and GID 0, asin 
the preceding example. There are other users with no login privileges, such as daemon. These generally 
vary from system to system, and you should leave them alone. "Nobody" is a special under-privileged user 
that some processes run as, because the nobody user cannot write to anything on the system. 


These special users are called pseudo-users. They can't log in to the system, but the system can start 
processes with their user IDs for security reasons. 


Note The/etc/passwd file syntax is special and fairly strict, allowing for no comments or blank lines. 


4.3.1Manipulating Users and Passwords 


Regular users interact with /etc/passwd with the passwd command. By default, the passwd command 
changes the user's password, but you can also use -f to change the user's real name, or -s to change the 
user's shell to one listed in /etc/shells. (Two other commands for changing the real name and shell are 
chfn and chsh.)passwd Is an suid-root program, running as the superuser when changing the 
/etc/passwd file. 


Changing /etc/passwd as the Superuser 


Because the passwd file is in plain text, the superuser may use any text editor to make changes. To add a 
user, you need to do nothing more than add an appropriate line and create a home directory for the user; 
deleting is the opposite. There is a special program called vipw that allows you to lock the passwd file 
and edit it as the superuser. To set another user's password, use passwduser as the superuser. 


However, most distributions frown on editing passwd directly because it is easy to make a mistake. For 
example, Red Hat Linux has the commands adduser and userdel to add and remove users. Even if you 
use these commands or vipw, though, you should still know about the password file format, because you 
may need to edit it in a pinch. 


4.3.2Working with Groups 


Groups in Unix offer a way to share files with certain other users on the system, but deny access to all 
other users. The idea is that you can set read or write permission bits (described in Section 1.17) for the 
group, but not the world (everyone else). This was once important because many users shared one 
machine, but this has become less significant in recent years as workstations have become more personal. 


As mentioned earlier, /etc/passwd has a group field. The /etc/group file defines the group IDs, as 
shown in this example: 


Foot? *:0*s7user 
daemon:*:1: 

bine * 324 

Sa toe 

adm: *:4: 
disk:*:6:juser,beazley 
NOGrOUp? *+65554- 
user r LUU 


The fields in /etc/group are as follows: 
a» [he group name. (This is what shows up when you run a command like 1s -1.) 


= The group password. This is hardly ever used; you can use * or any other default value. 


=» The group ID (a number). The GID must be unique within the group file. This number goes into a 
user's group field in their /etc/passwd entry. 


= An optional list of users that belong to the group. In addition to the users listed here, users with the 
corresponding group ID in their passwd file entries also belong to the group. 


Figure 4-2 identifies the fields ina group file entry. 


Group name 
Password 
Group ID 
| Additional members 


disk: *:6:juser, beazley 
Figure 4-2: An entry in the group file. 


To see the groups you belong to, run the groups command. 


Note Linux distributions have a convention of creating a new group for each new user with the same 
name as the user. 


4.4geitty and Login 


section 3.1.1 mentioned getty, a program that attaches to terminals and displays a login prompt. On 
most Linux systems, getty is uncomplicated because the system only uses it for logins on virtual 
terminals, with lines in /etc/inittab like this one for /dev/ttyl: 


1:2345:respawn:/sbin/getty 38400 ttyl 


In this example, 38400 is the baud rate. Virtual terminals ignore the baud rate; it's only there for backward 
compatibility with terminals connected to real serial lines. Some getty programs (like the mingetty that 
Red Hat Linux uses for virtual terminals) don't need the baud rate setting. 


You are probably most interested in the name of the file that getty prints as the login greeting: 
/etc/issue. This is a quick and satisfying way to deface your system without doing any real damage. 


After you type your login name, getty replaces itself with the login program, which then asks for your 
password. If you type the correct password, login replaces itself with your shell. Otherwise, you get a 
"Login incorrect" message. 


You now know what getty and login do, but it's unlikely that you'll ever need to configure or change 
them, because terminals on serial ports have been largely confined to the dustbin of history. The login 
program has an excessive number of options and configuration files, but they are mostly useless because 
they deal with security for network login methods, such as telnet, that are insecure in the first place. You 
will learn in Chapter 6 that you should not use them. 


Note If you want to send and receive faxes or dial up to your machine through your own modem, you 
may need to work with mgetty, an advanced getty that can work with fax and voice modems. 


4.5Setting the Time 


A well-functioning Unix machine depends on accurate timekeeping. The kernel is responsible for 
maintaining the system clock, the clock consulted when you run commands such as date. You can also 
set the system clock with the date command, but this is usually a bad idea, because you'll never get the 
time exactly right. You want your system clock to be very close to the correct time. 


PC hardware has a battery-backed "real-time" clock. It's not the greatest clock in the world, but it's better 
than nothing. The kernel usually sets its time based on this clock at boot time, and you can reset the 


system clock to the current hardware time with hwclock. If your hardware clock is in UTC (Universal 
Coordinated Time, or Greenwich Mean Time), run this command: 


hwelock =-hceosvys =——aLc 


You should try to keep your hardware clock in UTC to avoid any trouble with time zone or daylight savings 
time corrections. However, if you insist on keeping the hardware clock set to the local time, run this 
command instead: 


hwelock —-hctosys --Localtime 


Unfortunately, the kernel isn't terribly good at keeping time, and because Unix machines often stay up for 
months and years at a time, they tend to develop a drift. In addition, it's fairly rude to make the system 
clock jump ahead or back by running hwclock ona frequent basis; time-based system events can get lost 
or munged. If you want to gradually change the system clock based on the hardware clock, run this 
command: 


adjtimex --adjust 


However, this is only good for updating the clock manually when it has already drifted. To keep the time up 
to date, you need to run a network time daemon (see Section 4.5.2). 


4.5.1Time Zones 


The system clock runs in UTC. System libraries convert this into local time and compensate for daylight 
savings time and any other strange circumstances (such as living in Indiana). 


To set your system's time zone, make a symbolic link to the appropriate time zone file in 
/usr/share/zoneinfo from /etc/localtime. For example, here is the command to configure a 
United States Pacific Time Zone setting: 


in =-sf /usr/share/zoneinfo/US/Pacific /etc/localtime 


A long listing of the link should contain this output: 


7@ctc/localtime => /usr/share/zoneinto/US/Pacific 


lf you want to use a different time zone than the system default, you can do so by setting the TZ 
environment variable like this: 


$ export TZ=US/Central 
$ date 


4.5.2Network Time 


lf your machine is permanently connected to the Internet, you can run an NTP (Network Time Protocol) 


daemon and get the time from a remote server. The main NTP Web page is http:/Awww.ntp.org/, but if you 
don't really feel like reading through the mounds of documentation there, do this: 


1. 


2 
3: 
4 


Ask your ISP for the closest NTP time server. 
Put that time serverin /etc/ntpd.conf. 
Runntpdateserver at boot time. 


Runntpd at boot time. 


If your machine does not have a permanent Internet connection, you can use a daemon such as chronyd 
to maintain the time during disconnections. 


HINT You can also set your hardware clock based on the network time. First, set your system time 


from the network with ntpdate (or ntpd). Thenrun this command: 


hwelock -—=systonce -utc 


4.6Scheduling Recurring Tasks with Cron 


Unix provides a service called cron that runs programs repeatedly on a fixed schedule. Most experienced 
administrators consider cron to be vital to the system because it can perform automatic system 
maintenance. For example, cron runs log file rotation utilities. You should know about cron not only in 
the interest of knowing about everything that runs on your system, but also because it is just plain useful. 


You can run any program you like with cron, choosing the times that suit you. A program running through 
cron is called a cronjob. To install a cron job, you need to create an entry line in your crontab file, 
usually with the crontab command. Here is a typical crontab file entry that runs a command daily at 9:15 
AM: 


15 09 * * x /home/juser/bin/spmake 


The five whitespace-delimited fields at the beginning specify the schedule time (see also Figure 4-3): 
=» Minute (0—59) 
m Hour (0-23) 
a Day of month (1-31) 
=» Month (1-12) 


a Day of week (0-7; 0 and 7 are Sunday) 


r Minute 
Hour 
Day of month 
Month 
Day of week 
| Command 
15 09 + + * /home/juser/bin/spmake 


Figure 4-3: An entry in the crontab file. 


The preceding example says to run spmake daily, because the day of month, month, and day of week 
fields are all * (the * character matches all possible values, meaning every day of the month, every 
month, or every day of the week). If you want to run the job only on the 14th of each month, use this 
crontab line: 


15 09 14 * * /home/juser/bin/spmake 


You may select more than one time for each field. For example, if you would like the program to run on 
the 5th as well as the 14th, use 5, 14 in the third field: 


15 09 5,14 * * /home/juser/bin/spmake 


The remainder of the line is the command to run. When cron runs the job, the command may generate 
standard output or error or exit abnormally. If that happens, cron mails this information to you. You may 
want to redirect the output to /dev/nu11 or some log file if you find this annoying. 


The crontab(5) manual page is a full reference for the cron file format. However, there is also a crontab(1) 
manual page for the installation utility, so you'll need to use man 5 crontab to get to the right one. 


4.6.1Installing Crontab Files 


Each user can have their own crontab file. The system usually stores all crontabs in 
/var/spool/cron/crontabs. Normal users can't write to this directory. Instead, the crontab 
command installs, lists, edits, and removes a user's crontab. 


The easiest way to install a crontab is to put your crontab entries into a file, then use crontabfile to 


install fi 1e as your current crontab. The crontab command checks the file format to make sure that you 
haven't screwed anything up. If you want to list your cron jobs, run crontab -1. To remove the crontab, 
USs€crontab -r. 


Using temporary files to create and manipulate your crontab can be a little messy, so if you want to edit 
and install your crontab in one motion, run crontab -e. If you make a mistake, crontab tells you where 


the mistake is and asks if you would like to try editing again. 


4.6.2System Crontab Files 


Rather than use the superuser's crontab for scheduling recurring system tasks, Linux distributions normally 
have an /etc/crontab file. Do not use the crontab command to edit this file; it has a slightly different 
format with one more field — the user that should run the job. Here is an example for a job to run at 6:42 
AM, running as the superuser (root): 


42 6 * * * root /usr/local/bin/cleansystem > /dev/null 2>&1 


Some distributions also use the /etc/cron.d directory to store system crontab files. These files may 
have any name, but they have the same format as /etc/crontab. 


4.7Scheduling One-Time Tasks with At 


If you want to run a job in the future but don't want to use cron because the command should only run 
once, you can do it with the at service. For example, if you want to run my job at 10:30 PM, enter this 
command: 


at 22230 
my job 
End the input with a CONT ROL-D (at reads the commands from the standard input). 


To check that the job has been scheduled, use atq, or to remove it, use at rm. You also can schedule jobs 
days into the future by adding the date (in DD.MM.YY format; for example, at 22:30 30.09.03). 


There isn't too much else to the at command. at actually does not see much use in common practice. 
However, it can be handy for the odd time that you need to shut the system down in the future. 


4.8Tracking Individual Processes 


You saw how to use ps in Section 1.16 for listing process characteristics at any given time. ps is good for 
getting a snapshot of the current processes, but it does little to tell you how processes change over time. Using 
ps alone, it's not terribly easy to pick out processes that are using too much CPU time or memory. 


Thet op program displays the current system status and many of the fields that you might see ina ps listing, 
but it also updates the display every second. By default, top shows the most active processes (that is, those 
currently taking up the most CPU time). 


You can send commands to top with keystrokes. These are some of the most important commands: 
SPACEBAR Updates the display immediately 


M Sorts by current resident memory usage 

T Sorts by total (cumulative) CPU usage 

P Sorts by current CPU usage (the default) 

u Displays only one user's processes 

? Displays a usage Summary for all top commands 


If you want to know how much CPU time a process uses during its entire lifetime, use time. (If your shell has 
a built-in time command, you may need to run /usr/bin/time to get the full output shown below.) For 
example, to measure the CPU time used by 1s, run this command: 


time ls 


Afterls terminates, you should get some output like this: 


O.O05user 0.09system 0:00.44elapsed 31%CPU (Oavgtext+0avgdata Omaxresident)k 
Oinputs+Ooutputs (125major+5lminor)pagefaults Oswaps 


a User time is the number of seconds that the CPU has spent running the program's own code. On modern 
processors, some commands run so quickly that you may get a zero value here because the actual CPU 
time is so low that time rounds down to zero. 


a System time is how long the kernel spends doing the process's work (for example, reading files and 
directories). 


= Elapsed time is the total time it took to run the process from start to finish, including the time that the CPU 
spent doing other tasks. This number is not very useful for performance measurements. 


SeeSection 4.10 for a description of the other fields. 


4.8.1Finding Open Files with Isof 


Thelsof command lists open files and the processes that are using these open files. Because Unix places 
such a great emphasis on files, 1sof is among the most useful tools for finding trouble spots. 1sof does not 
stop at regular files — it can list network sockets, dynamic libraries, pipes, and more. 


Runninglsof onthe command line usually produces a tremendous amount of output. Here is a fragment of 
what you might see: 


COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 

init 1 root cwd DIR epi 4096 A J 

Init i root ted DIR Fpl 4096 2 d 

init 1 rooC TXL REG 3y l 24480 16214 /sbin/init 

Init 1 root mem REG eF 999542 32329 /lib/ld=2.3;2.50 
init 1 root mem REG 3,1 1251176 32208 /lib/libc-2.3:2.;50 
init 1 root 10u FIFO pdl 97634 /dev/initctl 


vi 3511 juser cwd DIR 


3/4 4096 163866 /home/juser/w 
vi 3511 juser rtd DIR Sy 4096 2 / 
yl 3511 juser txt REG 3,3 327048 561520 /usr/bin/vi 
vi 3511 juser 3rW REG 374 18993 163867 /home/juser/w/c 


The fields in this output are as follows: 


COMMAND The command name for the process that holds the file descriptor. 
PID The process ID. 
USER The user running the process. 


FD The file descriptor or the purpose of the open file. A file descriptor is a number that a process uses in 
conjunction with the system libraries to identify and manipulate a file. 


TYPE The file type (regular file, directory, socket, etc.). 

DEVICE The major and minor number of the device that holds the file. 
SIZE The file's size. 

NODE The file's inode number. 


NAME The filename. 


The Isof(1) manual page contains a full list of wnat you might see for each field, but you should be able to 
guess just by looking at the output. For example, look at the entries with cwd in the FD field. These lines 
indicate current working directories of the processes. Another example is the very last line, showing a file that 
the user is currently editing with vi. 


There are two basic approaches to running lsof: 


a List everything and pipe the output to a command like less, and then run a search to find what you're 
looking for. 


a Narrow down the list that 1sof provides with command-line options. 


If you use command-line options, you can provide a filename as an argument, and 1sof will list only the 
entries that match the argument. For example, the following command displays entries for open files in /usr: 


Isor /uūusr 


To list the open files for a particular process ID, run this command: 


lsor =p pad 


lsof has dozens of other options; 1sof -h provides a short summary. Most options pertain to the output 
format. Make sure that you look at Section 6.5.1 for 1sof network features. 


Note J sof is highly dependent on kernel information. If you upgrade your kernel, you may need to 
upgrade 1 sof. 


4.8.2Tracing Program Execution with strace and Itrace 


With the exception of time, the tools you have seen so far examine active processes. However, if you have 
no idea why a program dies almost immediately after starting up, even 1sof won't help you. You would have 
a difficult time even running 1sof concurrently with a failed command. 


Thestrace (system call trace) and lt race (library trace) commands can help you discover what a program 
attempts to do. These tools produce extraordinarily large amounts of output, but once you know what to look 
for, you will be in a good position for tracking down problems. 


Asystem call is a privileged operation that the kernel must perform for a process, such as opening and 
reading data from a file. st race prints all of the system calls that a process makes. To get the idea, run this 
command: 


strace cat. 7 dev/null 


The first part of the output deals primarily with loading shared libraries. You can ignore this stuff: 


execve ("/bin/cat™,. [car “sdev/null" |, 17 * 52. vars: 7/1): =-0 

brk (0) = 0x804b348 

open ("/etc/ld.so.preload", O _RDONLY) = -1 ENOENT (No such file or directory) 
open("/etc/ld.so.cache", O _RDONLY) =°3 

fstat64(3, {st _mode=S_IFREG|0644, st_size=14834, ...}) = 0 

old mmap (NULL, 14834, PROT READ, MAP PRIVATE, 3, 0) = 0x40015000 

close (3) = 0 

open ("/lib/libc.so.6", O_RDONLY) = 3 

rsads "\NITTEEP\E\VINTAONONONONONONONONONS\ONSGNONINOAONONZOOF\ I" se, 1024): =] 1024 


In addition, you can skip by all of the mmap output until you get to the lines that look like this: 


fstat64(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(l, 3), ...}) = 0 
open (" /dev/null", O_RDONLY|O_LARGEFILE) = 3 

fstat64(3, {st_mode=S_IFCHR|0666, st_rdev=makedev(l, 3), ...}) = 0 
brk (0) = 0x804b348 

brk (0x804c348) = 0x804c348 

brk (0) = 0x804c348 

brk (Ox804d000) = 0x804d000 

read (37. TM- 4096) = 0 

close (3) = 0 

close (1) = 0 

exit (0) = ? 


Now you see some interesting things. First, look at the open () call, which opens a file. The 3 as a result 
means success (3 is a file descriptor). Then, near the end, you see where cat reads from /dev/nul11 (the 
read () call — notice that the file descriptor is 3). Then there is nothing more to read, so the program closes 
the file descriptor and exits. 


So what happens when there's a problem? Try strace cat not_a_file instead and examine the open () 
call: 


open ("not_a_file", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or 
directory) 


Here, st race not only reports the error at the system-call level, but it also gives you a small description of the 
error. 


Missing files are the most common problems with Unix programs, so if the syslog and other log information 
isn't very helpful, and you have nowhere else to turn, st race can be of great use. You can even use strace 
on daemons that detach themselves. Here's an example: 


strace -o crummyd_strace -ff crummyd 


In this example, the -o option to st race logs the action of any child process that crummyd spawns into 


crummyd_strace.pid, where pid isthe process ID of the child process. 


Theltrace command tracks shared library calls. The output is similar to st race, but be warned, there are 
usuallymany more shared library calls than system calls. See Section 8.1.4 for more information on shared 
libraries (1t race doesn't work on statically linked binaries). 


4.9Adjusting Process Priorities 


It is possible to change the way the kernel schedules a process so that the process gets more or less CPU 
time than other processes. The kernel runs each process according to its scheduling priority, whichis a 
number between -20 and 19, with -20 being the foremost priority (yes, this can be confusing). 


Theps -1 command lists the current priority of a process, but it's a little easier to see how it works with 
thet op command, as shown here: 


Tasks: 79 total, 1 funnane, 77 sleeping, O stopped, 1 zombie 


CPUS): 9.4% us, ivOe SY, 0.4% ni, 88.4% id, 0.6% wa, 0.0% hi, 0.1% si 
Mem: 320364k total, 301420k used, 18944k free, 17320k buffers 
Swap: 514072k total, 13260k used, 500812k free, 68260k cached 
PID USER PR NI VIRT RES SHR S %CPU MEM TIME+ COMMAND 
23066 bra 16 0 1868 696 17324 R 5:0 U.S 0200.06 top 
T0382 ori Es 0 93288 74m 28m S L9 23:7 200704. 7/5 Mozillarirebird 
L -roor 16 O 1346 480 1316 5 0.0 0.1 0:04.28 Init 
2 root 34 19 O 0 0S 0.0 0.0 0:00.00 ksoftirgd/ 0 
3 root a = 10 0 0 US O40 0,0 0:06.36 events/0 
4 root P.O) 0 0 OS O40 0.0 0:08.89 kblockd/0O 
5 root 15 0 0 0 Co Unt Oal 0:00.02 khubd 
6 root Zo 0 0 O OS Oe. U0 0200.00 pdrlush 
7 root 15 0 O 0 0S 0:0 Ue<.O 1:29.48 pdflush 
8 root 15 0 0 O OS 0.0 0.0 1:01.23 kswapd0 
9 root 10 =10 0 0 0S 0.0 0,0 0:00.00 aio/0 
10. root 19 0 0 O US O40 0,0 0:00.00 scesi. eh. 0 
LT root 15 0 0 0 OS O20 O20 0:00.00 kseriod 
127 root 16 O 2412 5372 1360 $S 0.0 0-2 0:03.24 syslogd 
129 root 16 O 2344 464 1312 o OU. Gel 0:00.02 klogd 
134 daemon 19 O 1436 404 1388 S 0.0 0.1 0:00.00 portmap 
136 root 16 O- 1380 576 1352 5 0:0 Usd 0700.03 inetd 


In the top output, the PR column lists the kernel's current schedule priority for the process. The higher the 
number, the less likely the kernel is to schedule the process if others need CPU time. The schedule priority 
changes frequently during program execution according to the amount of CPU time that the process 
consumes. 


The schedule priority alone does not determine the kernel's decision to give CPU time to a process. Next 
to the priority column is the nice value (the NI column), a hint for the kernel's scheduler. This is what you 
care about when trying to influence the kernel's decision. The kernel adds the nice value to the current 
priority to determine the next time slot for the process. By default, the nice value is 0. If you want a certain 
process to take a backseat to other processes, running only when the other tasks have nothing to do, you 
can change the nice value to 19 with the renice command (where pid is the process ID of the process 


that you want to change): 


renice 19 pid 


This comes in handy when you're running some big computation in the background and don't want it to bog 
down your interactive session. 


If you're the superuser, you can set the nice value to a negative number, but this is almost always a bad 
idea because system processes may not get enough CPU time. In fact, you likely won't need to alter nice 
values much because most Linux systems have only a single user, and that user does not perform much 
real computation. The nice value was much more important in the days when there were many users ona 
single machine. 


4.10Monitoring System Performance 


Most Linux systems behave well under a distribution's default settings, and you can spend hours and days 
trying to tune your machine's performance without attaining any meaningful results. Sometimes 
performance can be improved, though, and this section concentrates primarily on memory and processor 
performance, and looks at how you can find out if a hardware upgrade might be worthwhile. 


The two most important things that you should know about performance on your system are the load 
average and the system's swap/page fault behavior. The load average is the average number of processes 
currently ready to run. The uptime command tells you three load averages in addition to how long the 
kernel has been running: 


up 91 days, ... load average: 0.08, 0.03, 0.01 


The three numbers here are the load averages for the past minute, 5 minutes, and 15 minutes. As you can 
see, this system isn't very busy, because the processor has been running at 1 percent capacity for the past 
quarter hour (because the last number is 0.01). Most systems exhibit a load average of 0 when you're 
doing anything except compiling a program or playing a game. A load average of 0 is usually a good sign, 
because it means that your processor isn't even being challenged, and you're saving power by not running 
a silly screensaver. 


If the load average is around 1, it's not necessarily a bad thing; it simply means that the processor has 
something to do all of the time — you can find the process currently using most of the CPU time with the 
top command. If the load average goes up near 2 or above, multiple processes are probably starting to 
interfere with each other, and the kernel is trying to divide the CPU resources evenly. (Unless, of course, 
you have two processors, in which case, a load average of 2 means that both processors have just enough 
to do all of the time.) 


A high load average does not necessarily mean that your system is having trouble. A system with enough 
memory and I/O resources can easily handle many running processes. Don't panic if your load average is 
high and your system still responds well. The system is just running a lot of processes. The processes 
have to compete with each other for the CPU, and therefore, they will take longer to do their computation 
than they would if there were one CPU for each process, but there's nothing to worry about. 


However, if you sense that the system is slow and the load average is high, you're probably running into 
memory problems. A high load average can result from the kernel thrashing, or rapidly swapping 
processes to and from swap space on the disk. Check the free command or /proc/meminfo to see how 
much of the real memory is being used for buffers. If there isn't much buffer memory (and the rest of the 
real memory is taken), then you need more memory. 


A situation where the kernel does not have a piece of memory ready when a program wants to use itis 
called a page fault (the kernel breaks memory into small chunks called pages). When you get a lot of page 
faults, it bogs the system down because the kernel has to work to provide the pages, robbing normal 
processes of their chance to run. 


Thevmst at command tells you how much the kernel is swapping pages in and out, how busy the CPU is, 
and a number of other things. It is probably the single most powerful memory-performance monitoring tool 
out there, but you have to know what the output means. Here is some output from vmstat 2, which 
reports statistics every two seconds: 


procs memory Swap 10 system Cpu 

r bw  swpd free buff cache si SO bi bo in ee us Sy id 
4 0 0 145680 3356 5812 89496 1 1 2 2 2 2 2 O O 
yO Q 145660 3328 5612 29490 O O O 0 102 38 O O 99 
D-0 0 145080 3320 5012 89496 O O O 42 111 44 al O g9 
O 0 0 145680 3528 3612 89496 O O O O 101 sis O 1 99 


The output falls into categories: procs for processes, memory for memory usage, swap for the pages 
pulled in and out of swap, io for disk usage, system for the number of times the kernel switches into 
kernel code, and cpu for the time used by different parts of the system. 


The preceding output is typical for a system that isn't doing much. You need to ignore the first line of 


numbers because it's incomplete and doesn't make sense. Here, the system has 145680KB of memory 
swapped out to the disk (swod) and 3328KB of real memory free, but it doesn't seem to matter, because 
thesi and so columns report that the kernel is not swapping anything in or out from the disk. (In this 
example, Mozilla is taking up extensive residence on the swap partition. Mozilla has a habit of loading a lot 
of stuff into memory when it is started, never bothering to actually use it.) The buff column indicates the 
amount of memory that the kernel is using for disk buffers. 


On the far right, you see the distribution of CPU time, under us,sy, and id. These are the percentages of 
time that the CPU is spending on user tasks, system (kernel) tasks, and idle time. In the preceding 
example, there aren't too many user processes running (using a maximum of 1 percent of the CPU), the 
kernel doing practically nothing, while the CPU is sitting around doing nothing 99 percent of the time. 


Now, watch what happens when a big, nasty program starts up sometime later (the first two lines are right 
before the program runs): 


procs memory Swap 10 system cpu 

r b w swpd free buff cache si SO DI bo in Gs. US Sy ad 
L -O U- -14098957 46068. T312 39980 O O O O LLG Eg O O- 100 
O-O (0) L40988 426608: T32 -58980 O O O O TOT Si O 0. 29 
L OF O 140988 3056 6780-56496 6 La, + LoaOG 14 Le 2159 323 o Aa 
T "Or 2AO9SsS 3304 S900. 56864 322 1268 100 268 T93 Ago. Al J 35 
0:0! 0: T4098 3496, 3642 SE1G0- -8G O BOY) O LEG. LTS 4 6 269 
1. *0i0. TAIS 3500. 36438 56248 O O 38 O ESO LOZ. LI 6 83 
2 3 Q 140988 3056: 4208 59588 42 1A. -2062 14 249 LSID: -20 6 74 
a QO 140464 S100 2000 65416 T6 96° “2598 LEG AS LLa So ak Ye 5 
Ae SO" O LAOS IZ. 280. ZALO Cosy O 14 4490 14 481 704 ii 5 94 
Za OD TAO - S056: 2428 “73076: 06 O 7184 68 SaD TTI O 9: “90 
210141176 4072 2112 81544 28 220 7314 Zoe SIA L748- 20° IY GI 
L2 -0 DAG SG 2050 L292 CUTZ Oi P9660 3532 L974 504 1054 2 a gi 
F0 TAS960! 30596- “L676 89864 O- 3044: . T458- -3096 490 876 1 6- 92 


The CPU starts to see some usage for an extended period, especially from user processes. The amount of 
buffer space starts to deplete as the new program requires new pages, and the kernel starts to kick pages 
out into swap space (the so column) to make space. Unfortunately, as the kernel does this, it has to wait 
for the disk to do its work, and the CPU sits idle while processes block (the b column), because they also 
have to wait for the memory pages that they need. You can see the effects of swapping all over the output 
here; more memory would have fixed this particular problem. 


This section hasn't explained all of the vmstat output columns. You can check on them in the vmstat(8) 
manual page, but you may have to learn more about operating systems first from aclass or a book like 
Operating System Concepts [Silberschatz]. However, try to avoid getting really obsessed with 
performance. If you're trying to squeak 3 percent more speed out of something and you're not working on a 
huge computation, you're probably wasting your time. 


4.11Running Commands as the Superuser 


Before you see any more system commands, you should learn more about how to run commands as the 
superuser. You probably already know that you can run the su command and enter the root password to 
start a root shell. This practice works, but it has these disadvantages: 


= You have no record of system-altering commands. 

=» You have no record of the users who performed system-altering commands. 
=» You don't have access to your normal familiar shell environment. 

=» You have to type the root password. 


Most larger workstation installations employ a package named sudo to allow administrators to run 
commands as root when they are logged in as themselves. For example, if you want to use vipw to edit 
the/etc/passwad file, you could do it like this: 


sudo vipw 


When you run this command, sudo logs this action with the syslog service under the local2 facility, so that 
you have a record of what you did. 


Of course, the system doesn't let just any user run commands as the superuser; you must configure the 
privileged users inthe /etc/sudoers file. As it turns out, sudo has a great many options (that you'll 
probably never use), and the side effect of this is that the /etc/sudoers file has a somewhat 
complicated syntax. Here is an example file that gives user1 and user2 the power to run any command 


as root without having to enter a password: 


User Alias ADMINS = useri , user2 
ADMINS ALL = NOPASSWD: ALL 


root ALL=(ALL) ALL 


The first line defines an ADMINS user alias with the two users, and the second line grants the privileges. 
TheALL = NOPASSWD: ALL part means that the users in the ADMINS alias can use sudo to execute 
commands as root. The second ALL means "any command." The first ALL means "any host" (if you have 
more than one machine, you can set different kinds of access for each machine or for groups of machines, 
but this book will not explain that feature). 


Theroot ALL=(ALL) ALL simply means that the superuser may also use sudo to run any command on 
any host. The extra (ALL) means that the superuser may also run commands as any other user. You can 
extend this privilege to the ADMINS users by changing the /etc/sudoers line shown earlier to this: 


ADMINS ALL = (ALL) NOPASSWD: ALL 


Note Use the visudo command to edit /etc/sudoers. This command checks for file syntax errors 
after you save the file. 


That's all you need to know about sudo for now. If you need its more advanced features, consult the 
sudoers(5) and sudo(8) manual pages. 


4.11.1Real UID and Effective UID 


When changing user IDs with programs like sudo, keep in mind that there is more than one active user ID 
in a running program. The ID that controls your access rights is the effective user ID. When you run a 
setuid program, Linux sets the effective user ID to the program's owner during execution, but it keeps 
your original user ID in what is called a real user ID. 


There is an additional saved user ID that is set to the program's owner when you run a setuid program. 
The idea is that a process can switch its effective user ID to the real or saved user ID during execution to 
perform different tasks as different user IDs. 


The reason that you might see different user IDs is that, by default, sudo changes the real user ID along 
with the effective and saved user IDs. Some programs don't like to have a real user ID of root. If you do 
not want sudo to change the real user ID, add this line to your /etc/sudoers file: 


Defaults stay_setuid 


Chapter 5: Configuring Your Network 


Overview 


Networking is the practice of sending data from one computer to another computer. For a simple 
connection to the Internet, you need to do the following: 


=» Connect a network interface on your machine to a network 
a Define a gateway to the rest of the Internet 
a» Show your machine how to resolve hostnames 


Unfortunately, there are so many different kinds of networks that there is no one simple formula to get your 
system talking to the rest of the world. For Ethernet connections with a static (fixed) IP address, a separate 
command or file performs each of the preceding steps. However, DHCP (Dynamic Host Configuration 
Protocol) configurations and PPP (Point-to-Point Protocol) connections use different configuration 
schemes. This chapter begins with setting up static Ethernet interfaces because they are easy to 
understand, and then it moves onto DHCP and PPP, including DSL (Digital Subscriber Line) and PPPoE 
(PPP over Ethernet) connections. 


After you know how to connect your machine to a network, you're ready to move on to more advanced 
topics, such as building your own networks and configuring firewalls, described later in this chapter. 


5.1 Network Layers 


If you really want to understand Linux network configuration, you must be able to distinguish each layer in 
the network. Here are the layers in the Internet, from the top level to the bottom level: 


Application Layer Contains the "language" that applications and servers use to communicate; 
usually a protocol of some sort. Common application layer protocols include Hypertext Transfer 
Protocol (HTTP, used for the World Wide Web), Secure Shell (SSH), and the File Transfer Protocol 
(FTP). 


Transport Layer Defines the data transmission characteristics of the application layer. This is host- 
specific information, and it includes data-integrity checking, source and destination ports, and 
specifications for breaking application data into packets. TCP (Transmission Control Protocol) is the 
most common transport layer protocol. 


Internet Layer Defines how to move packets from the source host to the destination host. The 
particular packet-transit rule set for the Internet is known as IP (Internet Protocol). This is sometimes 
called the network layer. 


Host-to-Network Layer Defines how to send packets from the Internet layer across the physical 
medium, such as Ethernet or a modem. This is sometimes called the physical layer. 


This may sound overly complicated, but it's important that you understand it, because your data must 
travel through these layers twice before it reaches its destination. Your bytes leave the application layer on 
the source host, then go down to the physical medium, across the medium, and then up again to the 
application layer on the destination host. 


Unfortunately, the layers sometimes bleed in strange ways, and terms like TCP/IP reflect the integration. 
The distinction is only getting more vague — in particular, devices that once only dealt with the Internet 
layer now sometimes look at the transport layer data to determine where to send data. 


To connect your Linux machine to the network, you need to concentrate on the Internet and host-to- 
network layers, so let's get straight to them. If you want to know a /ot more about layers (and networks in 
general), look at Computer Networks [Tanenbaum]. 


Note You have heard of another set of layers known as the ISO OSI (Open Systems Interconnection) 
Reference Model. This is a seven-layer network model often used in teaching and designing 
networks, but this book does not cover the OSI model because it is of little practical use for 
understanding how Linux Internet networking works. 


5.2 The Internet Layer 


The Internet as we know it is based on the Internet Protocol, version 4 (IPv4). The basic idea is that each 
computer on the Internet has a numeric /P address in the form of a.b.c.d, where a and d are one-byte 
numbers from 1 to 254, and b and c are one-byte numbers from 0 to 255. Technically, an IP address is just 
a four-byte (32-bit) number, but it's usually easiest to deal with the address as a dotted-quad sequence of 
decimal numbers, like 10.23.2.37, instead of something ugly like the hexadecimal 0x0A170225. 


Asubnet is a group of IP addresses in some (hopefully) regular order. For example, the hosts between 
10.23.2.1 and 10.23.2.254 could comprise a subnet, or all of the hosts between 10.23.1.1 and 
10.23.255.254 could also be a subnet. 


Subnets are important because they determine how your packets move between different machines. To 
communicate with another computer, your machine must know that other computer's IP address. If the 
destination is on your local area network's subnet, your system can send the data directly to that host, but 
if not, your machine probably has to send it to a gateway (or router) that transmits your data across 
another network on its way to the final destination. 


5.2.1More on Subnets 


Asubnet mask defines a subnet; for example, the subnet mask for 10.23.2.1- 10.23.2.254 is 
255.255.255.0. In pure binary form, the mask "covers" the bits in an address that are common to the 
subnet. For example, here are the binary forms of 10.23.2.0 and 255.255.255.0: 


LO0e2352 60" OUOU TOLO -OOOLOLTI JO0000LD: 00000000 
Va a 0) 6 Par -40 e wa oe LIT Lit Lo OOC00000 


Now, let's "cover" the bits in 10.23.2.0 with the 1s in 255.255.255.0 by marking these bits in bold: 


ol 0 eo wea Oe 00001010 00010111 Q0000010 O0000000 


Look at the bits that aren't in bold. You can set any number of these bits to 1 to get a valid IP address in 
this particular subnet (with two exceptions: all Os and all 1s). 


Putting it all together, you can see that a host computer with an IP address of 10.23.2.1 and a subnet mask 
of 255.255.255.0 can directly communicate with other computers that have IP addresses beginning with 
10.23.2 (to talk to the rest of the Internet, your computer needs to use a gateway). You can denote this 
entire subnet as 10.23.2.0/255.255.255.0. 


If you're lucky, you will deal only with easy subnet masks like 255.255.255.0 or 255.255.0.0, but you may 
be unfortunate and encounter stuff like 255.255.255.192, where it isn't quite so simple to determine the set 
of addresses that belong to the subnet. It helps to know that 192 is OxCO in hexadecimal, from which you 
should be able to figure out what addresses belong to such a subnet. If you aren't familiar with conversion 
between decimal, binary, and hexadecimal formats, you should probably learn now. The dc (desk 
calculator) program has excellent facilities for converting between different radix representations. 


A common shorthand for subnet masks is to identify the number of bits at the start of the subnet mask that 
have a value of 1. For example, 255.255.255.0 contains 24 1-bits to start the 32-bit sequence, so you can 
write the earlier example of 10.23.2.0/255.255.255.0 as 10.23.2.0/24. Table 5-1 shows several example 
subnet masks. 


Table 5-1: Subnet Masks 








Long Form Short Form 
255.0.0.0 8 
255.255.0.0 16 
255.240.0.0 12 
255.255.255.0 24 
255.255.255.192 26 











Therefore, to specify 10.0.0.0/255.0.0.0, you can use 10.0.0.0/8. 


5.3Basic ICMP Tools 


Before you configure your network devices, you should learn how to use some ICMP (Internet Control 
Message Protocol) tools. ICMP packets help you root out problems with connectivity and routing. 


ping (see http://Atp. arl.mil/~mike/ping.html) is one of the most basic network debugging tools. It sends 
ICMP echo request packets to a host. If the host gets the packet and feels nice enough, it sends an ICMP 
echo response packet in return. 


Let's say that you run ping 10.1.2.21 and you get this output: 


PING 10:1,2.21. ALO cl e2e2l) 2 DO data bytes 

64 bytes from 10.1.2.21: icmp_seq=0 ttl=255 time=8.0 ms 
64 bytes from 10.1.2.21: icmp_seg=1 ttl=255 time=3.2 ms 
64 bytes from 10.1.2.21: icmp_seg=2 ttl=255 time=3.4 ms 
64 bytes from 10.1.2.21: icmp_seg=4 ttl=255 time=3.4 ms 
64 bytes from 10.1.2.21: icmp_seg=5 ttl=255 time=3.2 ms 


The most important parts of the output are the icmp_seq number and the round-trip time. ping sends a 
sequence of echo request packets, one every second. 


Notice that there's a gap between 2 and 4 in this example. This usually means that there's some kind of 
connectivity problem. It is possible to get packets out of order, but if this happens, there's still some kind of 
problem because ping sends only one packet a second. If a response takes more than a second to arrive, 
the connection is extremely slow. 


The round-trip time is the total elapsed time between the moment that the request packet was transmitted 
and moment that the response packet arrived. If there are incomplete routes between the request source 
and the destination, ping immediately reports the ICMP "host unreachable" packets that come back as a 
result of the disconnection. 


On a wired LAN, you should expect absolutely no packet loss and very low numbers for the round-trip time 
(the preceding example output is from a wireless network). You should also expect no packet loss from 
your network to and from your ISP, as well as reasonable, steady round-trip times. 


Sadly, not all hosts on the Internet respond to ICMP echo request packets as they once did. Therefore, you 
may come across situations where you can connect to a Web site on a host, but not get a ping response. 


Another useful ICMP-based program is traceroute; it will come in handy when you reach the material 
on routing later in the chapter. Use traceroutehost to see the exact path your packets take to a remote 
host. One of the best things about traceroute Is its reporting of return-trip times at each step in the 
route, as demonstrated in this output fragment: 


A 206.220.243.106 1:163 mse 04997 ms 1.182 ms 

5 4524.203405 1.312 ms €1.12 ms 1.463 -ms 

6- 64.1539., 1,;223 Tađdal me 1.37 ms 1-347 ms 

T O4.1 59.0630 8546402 ME 5:025 MS 253603 ME 

o 209.247.102.250 55.09 ms 55.617 ms 53:904. ms 
9 209.244.14.226 55.851 ms 55.726 ms 55.832 ms 
10 209.246.259.174 56.419 məs 56.44 ms 56.423 ms 


Because this output shows a big latency jump between hop 6 and hop 7, that part of the route is probably 
some sort of long-distance link. 


You can put these ICMP tools to use when setting up a working network interface, as the next few sections 
will show you how to do. 


5.4Configuring Interfaces and the Host-to-Network Layer 


On a Linux system, you connect the Internet layer to the physical medium, such as an Ethernet network or 
a modem-based connection, with a networkinterface. Common network interface names are ethO (the 
first Ethernet card in the computer) and pppo (a PPP interface). 


The most important command for viewing or manually configuring the network interface settings is 
ifconfig. To see your current interface's settings, run this command: 


iitconiig =a 


You do not need the -a in Linux, but other Unix variants require this option. The output should look 
something like this: 


etho Link encap:Ethernet HWaddr 00:40:05:A0:7F:96 
inec addii lo, less Beasts TOs b.24259 Maskt 2554255, 25530 
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
RX packets:806961 errors:1 dropped:0 overruns:0 frame:0 
TX packets:811658 errors:0 dropped:0 overruns:0 carrier:0 
collisions: 0 
RX bytes:726765161 (693.0 Mb) TX bytes:110229902 (105.1 Mb) 


lo Link encap:Local Loopback 
inet addre 127.00.) Mask?255.0,./0.0 
UP LOOPBACK RUNNING MTU:16436 Metric:1 
RX packets:44 errors:0 dropped:0 overruns:0 frame:0 
TX packets:44 errors:0 dropped:0 overruns:0 carrier:0 
cōõllisionsiU 
RX bytes:3569 (3.4 Kb) TX bytes:3569 (3.4 Kb) 


The left side contains interface names, and the right side contains the settings for each interface. You can 
see that each interface has an IP address (inet addr) and a subnet mask (Mask), but you should also 
take careful note of the lines containing UP and RUNNING, because these tell you that the interface is 
working. 


Thelo interface is a virtual network interface that is called the loopback because it "loops back" to itself. 
127.0.0.1 is the IP address of localhost, so connecting to this address is the same as connecting to the 
machine that you're currently using. 


Your system calls ifconfig from one of its init.d scripts at boot time to configure the 10 loopback 
interface. It's the only part of the network that is actually the same on any Linux machine, so it's a great 
place to start when you're trying to figure out how your particular distribution sets up networks. For 
example, in Red Hat Linux, each network interface has a script in /etc/sysconfig/network- 
scripts. You should be able to find the loopback device configuration by digging around in /etc with 
grep ifconfig. 


If you have a static IP address on an Ethernet interface, your system's boot sequence should set up the 
interface in a manner very similar to the loopback. However, you can manually configure an IP address 
and netmask for an Ethernet network interface named ethno with this command: 


ifconfig ethO address netmask mask 


The preceding command allows your machine to talk to every other host in the subnet defined by 
address and mask, but it does not let you go beyond the subnet, because you have not supplied a default 


gateway (gateways will be explained in the next section). 


If you do not connect your system to the network with a static IP address on an Ethernet network, but 
rather, have a link such as a PPP or PPP-over-Ethernet (PPPoE) DSL connection, or if you use DHCP to 
get host information, you do not configure your interface with ifconfig (see Sections 5.7,5.8, and 5.9 for 
those cases). However, even with those other types of connections, ifconfig -a is very useful for 


debugging. 


5.5Configuring a Default Gateway 
In the previous section, you saw how to manually configure an Ethernet network interface with a particular 


address and mask. You can connect to networks outside your subnet by adding a default gateway to your 
Kernel's IP routing table with this command: 


route add default gw gw-address 


Thegw-address parameter is the IP address of your default gateway; it must be an address in a locally 
connected subnet defined by the address and mask settings of your network interface (as described in 


theprevious section). If you have a static IP address, a default route setting usually goes along with an 
ifconfig command in your boot sequence. For other kinds of connections, other programs usually set 
the default route. 


To view the current default route, run this command: 


route -n 


The output should look something like this: 


Kernel IP routing table 


Destination Gateway Genmask Flags Metric Ref Use Iface 
Oe eee 0.0.0.0 2502 oda aNs U O O O eth0 
O00) lOe Me Ze alk 020.00 UG O O O eth0 


Theifconfig command that configured the etho0 network interface in this example added the first route 
line. However, the last line is the default gateway, because it has a netmask of 0.0.0.0, which allows the 
host to reach all other hosts on the Internet. The gateway's IP address is 10.1.2.1, and you can see that 
you reach that gateway through the eth0O network interface. 


If you mistype a gateway address when adding a route, you cannot attach another gateway until you delete 
the erroneous entry. To remove the current default gateway, run this command: 


route del -net default 


Figure 5-1 onthe next page shows a typical local area network with a gateway, or router. The subnet is 
10.1.2.0/255.255.255.0 with a default gateway of 10.1.2.1. Each IP address represents a network interface, 
as does the uplink IP on the gateway. If you do not run the gateway, you do not need to worry about its 
uplink — for example, if you are configuring a host with an IP address of 10.1.2.4, you need only know that 
10.1.2.1 is the local IP address on the gateway. 


jie 10. 1.2.0/255.25 5,255.0 





Figure 5-1: A typical local area network with an Internet link. 


Note If your netmask is 255.255.255.0, your router is probably at node 1 of the subnet (for example, 
10.1.2.1 in 10.1.2.0/255.255.255.0). This is convention, not necessarily reality. 


A properly configured default gateway allows you to connect to the rest of the world. However, you still 


need to Know how to convert common names, or hostnames, to IP addresses. 


5.6Resolving Hostnames 


The last bit of configuration for a simple network connection is hostname resolution. Simply put, your 
computer needs to know how to translate a name such as www.example.com to an IP address. Without 
this capability, your Internet connection is practically worthless — no one in their right mind advertises IP 
addresses for Web sites and email addresses, partly because a host's IP address is subject to change, but 
also because it's not easy to remember a bunch of numbers. 


The most basic way to do this mapping is with the /etc/hosts file, which looks like this: 


127.0.0.1 localhost 
10.1.1.3 atlantic.aem7.net atlantic 
lOs ledeg pacific.aem7.net pPaciriic 


In the bad old days, there was one central hosts file that everyone repeatedly copied to their own 
machine to stay up to date. This might have worked for a handful of machines, but as the 
ARPANET/Internet grew, this quickly got out of hand. The Domain Name System (DNS) decentralized 
hostname resolution. The idea is that your machine's resolver (in the C library) asks a nameserver for the 
IP address corresponding to a hostname. If the nameserver does not know, it can find out who is 
responsible for that name and ask for the IP address from the responsible party's nameserver. If you do 
not already know the DNS server addresses that your ISP uses, ask your provider for them. 


Let's say that your ISP's nameserver addresses are 10.32.45.23 and 10.3.2.3. Put the servers in your 
system's/etc/resolv.conf file like this: 


search mydomain.example.com example.com 
nameserver 10.32.45.23 
nameserver 10.3.2.3 


Thesearch line in the preceding example asks the resolver to look for the IP address of 
host.mydomain.example.com, and for host.example.com if you ask for an incomplete domain 
namehost that does not resolve on the first attempt. 


For a simple network configuration where you do not have control of your own nameserver, there is little 
else to know about DNS except how to make sure that your machine's hostname resolution works. To 
verify this, type host known_name where known_name is some Internet name that you know exists 


(such as www.example.com). The output should be something like this: 


known_name has address 10.218.44.5 


You should also be able to look up a name based on its IP address with hostaddress. If the address 
owner is doing their part, you should get the hostname corresponding to address, but DNS administrators 
often do not configure their servers to return these hostnames. Furthermore, the mapping of hostname to 
address is not one-to-one; you can have many hostnames for a single address. 


Before moving on, take a look at /etc/nsswitch.conf. It should have a line like this: 


hosts: files dns 


Puttingfiles ahead of dns here ensures that your system checks the /etc/hostsfile for the hostname 
of your requested IP address before asking the DNS server. This is usually a good idea (especially for 
looking up localhost), but your /etc/hosts file should be as short as possible. Don't put anything in there 
because you want to boost performance; it will burn you later. You can put all the hosts within a small 
private LAN in /etc/hosts, but the general rule of thumb is that if a particular host has a DNS entry, it 
has no place in /etc/hosts. 


Note The/etc/hosts file is also useful for resolving hostnames in the early stages of booting, when 
the network may not be available. 


DNS is a fairly broad topic, and if you have any responsibility for domain names, have alook at DNS and 
BIND [AlbitZ]. 


5.7Using DHCP Clients 


The Dynamic Host Configuration Protocol (DHCP) allows a host to get its IP address, subnet mask, default 
gateway, DNS servers, and other information from the network, so that you don't have to type these 
parameters by hand. Network administrators like DHCP because users don't have to nag them for IP 
addresses when they want to connect to the network. 


Likei £config, DHCP works on a network interface name. All Linux distributions have a network interface 
setup option for DHCP, so if you don't feel like digging around to try to find out what file controls the 
interface settings, it's all right to let your distribution's setup program do the work. 


When your machine asks a DHCP server for an IP address, it is really asking for a lease on an address for 
a certain amount of time. When the lease is up, your machine can ask to renew the lease. 


Most distributions use the ISC (Internet Software Consortium) dhclient program to request and retrieve 
IP address information from a DHCP server. To get a lease, you can run dhclient by hand on the 
command line, but before doing so you must remove any default gateway route. Once you have done that, 
you can request an IP lease by running the following command: 


dhclient eth0O 


dhclient stores its process ID in /var/run/dhclient.pid and its lease information in 
/var/state/dhclient.leases. 


Red Hat Linux versions 7 and earlier use a program called pump instead of dhclient. You can run it with 
pump -i interface, but you may want to consider getting dhclient from ISC or as an RPM. 


5.8PPP Connections 


So far, you have seen Internet connections based on a local area network (LAN) and a default gateway on 
that network. However, if you have a modem and telephone line to connect to the Internet, you cannot use 
the same host-to-network layer interfaces described in previous sections because your connection does 
not run over the Ethernet physical medium. 


The most popular standard for single-machine Internet connectivity with a modem is PPP (Point-to-Point 
Protocol). Although PPP is avery flexible protocol, its main use is to connect one machine without a direct 
IP connection to second machine (a PPP server) that has one, with the second machine possibly asking 
the first for a username and password along the way. 


When you dial up and register with a PPP server, the server gives you your own IP address and tells you 
about its IP address. After the connection initiates, the PPP software on your machine knows how to reach 
the server's IP address, and the server knows how to reach your machine's IP address. To reach the rest 
of the Internet, you set the default gateway to the server's IP address and let the server do the rest. Figure 
5-2 shows the link between your local IP address and the server's IP address. As with the routers in LANs, 
you do not need to worry about the server's uplink IP address. 
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Figure 5-2: A typical PPP link. 


Although it's nice to know how this process works, you are probably more interested in actually getting a 
connection to work. Specifically, you want to know the answers to these questions: 


=» How do | get my modem to dial the server's number? 
a» Where do! put my username and password? 
=» How do | start and stop a network interface like ppp0? 


The key to the whole mess is a pppd, a system daemon that acts as an intermediary between a serial port 
on your machine and a PPP network interface such as ppp0. To set up everything, you need to do four 
things: 


1. Verify that your serial port, modem, and login (username and password) work. 
2. Create an options file in /etc/ppp/peers containing configuration options. 

3. Create a chat script to make your modem dial out (and possibly log in). 
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. Set up Password Authentication Protocol (PAP) or Challenge Handshake Authentication Protocol 
(CHAP) user authentication if necessary. 


These four steps are explained in the following sections. If your distribution's dial-up configuration tool 
works for you, it may not be worth your while to read through these sections. However, if you don't read 
through them, at least run ifconfig when you're connected to see what the PPP interface name is, 
because you may need it later in the chapter for Network Address Translation (NAT) or firewalling. 


5.8.1Testing Your Serial Port and Modem 


Before you start with the full pppd setup, you should use minicom to make sure that your modem works 


and that you can initiate a PPP connection. Modems talk to your computer through serial port devices, so 
the first thing you need to do is determine the serial port that your modem is connected to. 


Note Don't try to use a Winmodem with Linux. See Section 17.2.4. 


As mentioned in Section 2.3.2, Linux serial port devices are at /dev/ttyS*. If you're having problems, 
it's best to check your boot messages as described in Section 4.1 to see if the modem setup looks right 
and to verify that you're using the correct serial port device. In addition, you can run cat 
/proc/ioports for more port statistics. Most modern hardware and kernels actually recognize the 
hardware configuration correctly, but if you have extra serial ports on different hardware interrupt numbers 
(IRQs), you may need to run setserial to specify the hardware parameters that the serial port uses. For 
example: 


setserial /dev/ttyS3 irg 5 


See the setserial(8) manual page for more options and a good overall explanation of serial ports. 


Now you can get down to the business of checking the connection to the dial-up server with these steps: 
1. Configure the baud rate, serial port, and other minicom settings with this command: 


minicom -s 


2. After choosing your settings, exit the configuration menu. minicom then attempts to connect to the 
serial port. 


3. Upon success, you should be able to enter AT commands. Test it by typing AT and then pressing 
ENTER. You should get OK as a response. If you're having trouble, skip to step 7 and then back to 
step 1. 


4. Reset your modem (that is, clear its volatile data) and dial your ISP's number by typing these two 
modem commands: 


atz 
atdtnumber 


5. Your modem should connect with a CONNECT message. W ait five or ten seconds after you see the 
CONNECT message. You're looking for "garbage" like this: 


~V}#A!}F!FL} F.} SPE} } 


If you get this junk immediately after you see the CONNECT message, you've got a PPP connection, 
and furthermore, you probably need to use the PAP or CHAP authentication system when dialing 
up. Make a note of this and skip to the last step. 


6. If you get something like Username: instead, type in your username (and password, if necessary). 
After logging in, keep going (you might need to type ppp or something similar) until you get the 
"garbage" mentioned in the previous step. In this case, you probably do not need to use PAP or 
CHAP. 


7. Exitminicom by pressing CONTROL-A, then x. If minicom hangs when you're trying to exit, you 


can wait a little while to see if it times out, or you can be impatient, open another window, and use 
kill -KILL onthe minicom process. 


When interacting with the modem and remote server, take careful notes of all the settings you used, what 
you had to type, and the responses you got back. You'll need this information to write a chat script. 


5.8.2Starting pppd with Options Files 


Now that you know how to log in to your ISP's server manually, you can create a configuration file 
containing the same connection settings you used with minicom -s. This allows you to make pppd dial 
the telephone number automatically. 


Pick a name for the ISP and create an options file called /etc/ppp/peers/name, with the following 
lines: 


toyol dLS200 ecresccs 

connect '/usr/sbin/chat -v -f /etc/ppp/chat-name' 
noauth 

defaultroute 

mru 1500 


The first line specifies the modem device, the CPU-to-modem speed, and the hardware flow control. You 
will read about these options in Section 5.8.5. Don't bother to tweak anything yet; for now, just make sure 
that your pppd can start properly. 


Now make a chat file named /etc/ppp/chat-name. This chat file determines the connection 


commands sent through the serial line and the responses that you expect from your ISP. For now, just fill it 
with the following lines, where number is your ISP's telephone number: 


ABORT "NO CARRIER" 

ABORT BUSY 

"H ATA OK 
ATDTnumber CONNECT 


Try to establish a connection by starting pppd with this command: 


pppd call name 


Monitor the progress by looking at your system log. The pppa syslog facility name is daemon, so the log 
information is probably in /var/log/daemon.log or something similar. However, the chat program 
used for initializing the modem connection may use a different facility, such as local2. 


You should get messages like this in your log file: 


Jul 29 18:40:46 mikado pppd[o634 


[ : pppd 2.4.1 started by root, uid 0 
Jul 29 18:40:47 mikado chat[635 

[ 

[ 


] 

]: [chat messages] 
Jul 29 18:40:47 mikado pppd[634]: 

Jul 29 18:40:48 mikado pppd[634] 


Connect script failed 
Exit. 


Take a careful look at the chat messages. They should tell you if the modem is dialing up, and if it gets 
an answer and connection from the remote server. 


Even if the preceding connection settings work so far, expect the configuration to fail because you haven't 
supplied a username and password. Watch the failure in the log carefully, because it tells you how to 
proceed. For example, if the logfile says Connect script failedasinthe preceding example, then 
you have a problem with the dial-up portion (in addition to not having supplied a username and password). 


In particular, look out for these types of failures: 


a |/O errors happen when your serial port and modem are not set up correctly. Double-check the first 
line of the options file to make sure it matches the settings that you verified earlier with minicom. 


a If the remote system uses PAP or CHAP, expect an authentication error. Skip to Section 5.8.4 to see 
how to configure PAP and CHAP. 


=» Timeouts generally go along with incomplete chat scripts. Proceed to the next section to see how to 
finish your script. 


=» Badchat output, suchas NO CARRIER and BUSY, are signs of telephone line problems. Double- 
check the ISP's telephone number. 


To stop a PPP daemon after it finishes its chat sequence, use the following command, substituting the 
appropriate network interface name if yours is not ppp0: 


kill ‘cat /var/run/ppp0.pid- 


5.8.3Chat Scripts 


Achat script is a file containing commands that go out through the serial line and responses that you 
expect to receive from the serial line (the responses come from the modem or ISP, depending on the 
current stage of the dial-up process). Recall this chat script line from the previous section: 


ATDTnumber CONNECT 


This means that the chat program should dial with the ATDTnumber modem command and expect a 
response of CONNECT. After receiving a CONNECT response, chat moves to the next thing in the chat file. 
After stepping through the entire file, and assuming that everything goes as anticipated, chat returns 
success and pppd proceeds with a PPP connection. 


The details of your chat script depend on your ISP, and you should be armed with the information you got 
inSection 5.8.1 before trying to complete your script. However, here is a script that works for most service 
providers that use Cisco equipment (where username is your username and password is your password): 


ABORT "NO CARRIER" 
ABORT BUSY 

mY ATG OR 
ATDTnumber CONNECT 
"Ww ername: 
username ssword: 
\gpassword > 


PPP 


TheABORT keywords and strings at the top of the script cause chat to exit with an error if it encounters 
one of those strings. In addition, \qstring tells chat not to send string to syslogd. In this case, it 
tellschat not to send password to syslogd when making a record of the connection. 


Remember that if you use PAP or CHAP, your chat script likely should not contain any username or 
password information. However, if your chat script does contain username and password details (like the 
preceding script), you should be mostly done after you finish the script — try to verify the connection with 
ifconfig -a, and try to access the Internet. If everything works, skip to Section 5.8.5 for information on 
how to tweak your options file. 


Note For more information on chat, see the chat(8) manual page. 


5.8.4P AP and CHAP 


Many dial-up ISPs require that logins use PAP (Password Authentication Protocol) or CHAP (Challenge 
Handshake Authentication Protocol). If your ISP has such a requirement, get started by adding the 
following entry for your username to the end of the /etc/ppp/peers/name options file from Section 


5.8.2: 


nameusername 


Warning Don't put require-pap in your options file. This option is primarily for running a PPP 
gateway; it requires the remote system to authenticate with your system. 


PAP 


If your ISP uses a PAP server, edit /etc/ppp/pap-secrets. The lines in that file have this format: 


your_hostname remote_hostname password ip_address 


The references to hostnames are primarily for use by PPP servers. For a simple client machine, you can 
put this in pap-secrets: 


username ü password j 


The* means to accept any remote hostname. Don't bother with the IP address, either, because the server 
should give you one. 


The preceding instructions will work if you have only one ISP. However, if you have multiple ISPs, do the 
following for each: 


1. Pick a name for the ISP (isp_name). 


2. Add the following line to the ISP's options file in /etc/ppp/peers: 


remotenameisp_ name 


3. Add the following to your pap-secrets file: 


username isp_name password a 


CHAP 


If you run into a CHAP server, proceed as you would with a PAP server and multiple ISPs, but enter the 
username and password into /etc/ppp/chap-secrets in the two-line format that follows: 


username isp_name your_password 
isp_name username their password 


5.8.5Options Files 


The options files in /etc/ppp/peers contain valid pppd command-line options. Therefore, you could run 
pppd at the shell prompt with these options as command-line arguments, and indeed, some distributions 
(like Red Hat Linux) use scripts to set up big pppd invocations, completely bypassing /etc/ppp/peers. 
However, it's far more convenient to use options files because they enable you to invoke pppd call 
file to activate the connection. 


Note In addition to the per-connection file in /etc/ppp/peers, you need a /etc/ppp/options file 
for global settings that apply to all pppd processes. This file must exist, even if it's empty. 


As mentioned earlier, the first two options in the options file are the serial device (with or without /dev/ at 
the front) and the serial port soeed. For most serial ports and modems, 115200 is an appropriate speed, 
but you may wish to lower it to 57600 or lower for really slow systems. Recall that the following was the 
first options line in Section 5.8.2: 


ttCyol L15200 grtacts 


All other pppd options follow the serial port and baud rate. Here are some of the most important pppd 
options: 
ertscts Enables RTS/CTS hardware flow control (popular with PC hardware). 


defaultroute Sets the kernel's default route to the machine on the other side of the PPP 
connection. Don't forget this option; otherwise, you won't be able to reach anything on the Internet 
beyond the host on the other end of the PPP connection, which usually isn't very useful. If you happen 
to forget this when testing, you can manually add a route with the route command (see Section 
3.11). 


noauth Specifies that the remote machine should not send you a username and password; this is 
another option that you don't want to forget. 


connect command Initializes the serial port with the specified command, usually a chat invocation 
like this: 


connect '/usr/sbin/chat -v -f /etc/ppp/chat-name' 


nodetach Run pppd in non-daemon mode; this prevents the shell prompt from returning immediately 
after you run pppd from the command line. This is handy for debugging, because you can press 
CONT ROL-C to interrupt a connection. 


mrusize The maximum receive unit option tells the remote server not to send any chunks of data 
larger than size. The default is 1500 bytes; 3000 works for faster connections. Do not set this (or 
mtu) to a number lower than 1300, because many applications prefer that their data be sent in larger 
chunks. 


mtusize The maximum transmission unit option requests that your machine not send (transmit) any 
chunks of data larger than size tothe remote server. The default is 1500 bytes. The quality of your 
telephone line or switch often determines how high you may set size. 


namename Forces pppd to use name as the username (or local system name) for PAP or CHAP 
authentication. 


remotenamename Forces pppd to use name as the remote system name for PAP or CHAP 
authentication. 


debug Activates verbose syslog debugging. 


Note If your connection frequently hangs or drops, and the ARQ light on your modem blinks often, you 
probably have a poor switch or telephone line. If you hear any "static" when you pick up your 
telephone (especially before making a call), your telephone company may be overloading your 
local phone switch. See if they will put you on a better switch. It's also possible that there's a 
problem between your phone jack and the phone network box in your building. 


The pppd(8) manual page has many more options. Keep in mind that most options are only relevant when 
you're running a PPP server, not a client. To be entirely truthful, there is no technical difference between a 
PPP server anda PPP client. They are simply peers that know how to contact each other. However, when 
one of the peers acts as a gateway to the rest of the Internet, you often say that this peer is the server. 


5.8.6PPP Directory Permissions 


The/etc/ppp directory should be mode 700; that is, the superuser should have read-write-execute 
access, and no other users should have access. 


However, any user can run pppd call name if that user has read access to the chat script and pppd is 
setuid root. Some distributions do not trust pppd and do not have the setuid bit set by default. As with any 
program, the setuid bit is a certainly a matter of concern if you don't trust your own users, but it's not so 
critical if you're the only user on the system. 


There is no standard for which user should start PPP (or even how to set up the connection). PPP has 
always been somewhat clumsy to operate because the Internet grew up around networks that were always 
active. Broadband connections solve this problem, though not everyone has access to them (or wants 
them). 


5.9Broadband Connections 


The term broadband vaguely refers to connections faster than a conventional modem link. For most users, 
this means a DSL or cable modem connection. One of the most frustrating things about broadband 
providers is that they don't tend to tell you what kind of technology they use. 


Most DSL or cable modems have an Ethernet port, requiring you to have an Ethernet interface on your 
computer, and as you saw in Section 5.4, it's not hard to set up an Ethernet interface (in theory, at least). 
Unfortunately, not all providers send straight IP from the Ethernet port, so things can get more 
complicated. 


For the most part, your setup depends on the kind of broadband connection you have. These rules 
generally apply: 


Cable modem These devices usually speak straight IP, and you can do the network configuration with 
DHCP. Plug your Linux machine into the cable modem and look at Section 5.7. If you have a static 
(fixed) IP address, you should be able to set up the connection by hand as described in Section 5.4. 
You may need to perform some configuration on the cable modem before it will connect to the outside 
world, but that's usually done with a Web browser. 


DSL with a dynamic IP address DSL providers usually send PPP over Ethernet (PPPoE) through 
their devices. This is the most common kind of DSL connection, and as luck would have it, it's the 
most complicated. See Section 5.9.2 for more information on PPPoE. 


DSL with a static IP address Some DSL providers talk straight IP over Ethernet if you have a fixed 
IP address. In this case, you can use the manual configuration described in Section 5.4. 


The preceding rules are based on the hardware available to service providers, but your provider may be 
different. Telephone companies prefer PPPoE because it's easier for them to monitor and authenticate, 
but cable companies tend not to care. You can probably determine the type of connection the ISP provides 
by looking at the setup instructions for Windows. If you see anything in the Windows screenshots with 
"pppoe" in the name, it's probably a PPP-over-Ethernet connection. 


Note If a broadband service forces you to use a modem or another device that doesn't have an 
Ethernet port, you will probably have a very difficult time getting it to work under Linux. Evaluate 
alternative ISPs. 


Don't expect your ISP's technical support to be very helpful to anyone running Unix. It doesn't matter that 
they're using technology that came straight from Unix machines — you're a fringe user, meaning that you 
have to figure it out yourself. 


5.9.1Routers 


Before you connect your Linux machine directly to a broadband connection, you need to ask yourself this 
very important question: /s it really worth it? 


The alternative is to purchase a small router. These very inexpensive devices not only speak straight IP 
and PPPoE, but often include a multi-port 10/100Base-T Ethernet switch and a wireless access point. They 
can do network address translation, route packets from the outside world to ports on their internal 
networks, and more. (Sometimes they even have pretty lights that serve as eye candy when you have 
nothing else to do.) 


Routers are especially attractive for Linux users with PPPoE connections. PPPoE Is a hassle, because you 
have to worry about several configuration files and daemons, not to mention possible trouble when 
upgrading. If you put a router between a DSL connection and your Linux machine, you can configure your 
machine's IP address statically or with DHCP. 


You can also build a network with a router. Their built-in switches not only share the broadband connection 
with all of the machines on the network, but also link those machines with each other. 


Of course, a Linux machine with two Ethernet cards and a hub or switch can do anything that a router can 
do. Infact, you have much finer control over a full-blown Linux machine acting as a router. However, very 
few people actually have a use for the fancy tricks you get with Linux, and the cost of a router is not much 


more than the cost of a switch. 


If you decide to purchase a router, skip to Section 5.10 for Ethernet networking details. Otherwise, if you 
are using PPPoE to connect to your ISP and you want to do everything on your Linux machine, continue 
reading. 


5.9.2PPP Over Ethernet (PPPoE) 


PPPoE support in Linux is still in a somewhat experimental stage. To use it, get the rp-pppoe package 
fromhttp:/Avww.roaringpenguin.com/ to extend pppd to full Asymmetric Digital Subscriber Line (ADSL) 
support. You also need a reasonably recent version of pppd, such as 2.4.1. All of this might come with 
your distribution, but as with most new software, it's a good idea to compile your own set from source code 


(seeChapter 9). 


The easiest way to set up rp-pppoe is to run adsl-setup as root. This creates appropriate 
/etc/ppp/pppoe.conf and /etc/ppp/pap-secrets files for your machine. If you read through 
pppoe.conf, you'll find only two critical settings: 


ETH=interface 


USER=username 


Most computers have only one Ethernet card, so you'll probably use et hOas the ETH setting. USER Is 
your username at your ISP. This must match the settings in pap-secrets (see Section 5.8.4 for 
information on PAP). 


To manually start the connection, run this command: 


adsl-start 


After you verify that everything works correctly, you can put adsl-start In an init.d script to activate 
the connection at boot time. adsl-start is a shell script that processes pppoe.conf and then runs 
asdl-connect, which in turn runs a big, ugly pppd command line. In principle, you could put all of this 
into an options file, but itis almost certainly not worth the hassle. 


To stop the connection, use adsl-stop. 


Warning Back up your /etc/resolv.conf if you put any kind of work into it, because ads1- 
connect may replace your DNS settings. 


5.10Ethernet Networks 


Ethernet is by far and away the most popular physical medium for local area networks. Although there 
have been several kinds of cables, cable topologies, connectors, and speeds in Ethernet history, the basic 
hardware, protocol specification, and programming techniques have proved remarkably robust. Figure 5-3 
on the next page shows the physical appearance of an Ethernet network. 
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Figure 5-3: A twisted-pair Ethernet local area network. 












Ethernet networks have always been cheap, but the unshielded twisted-pair (UTP) devices that gained 
popularity in the 1990s are now inexpensive almost to the point of absurdity. Many new computers (and 
devices such as video game consoles) have built-in Ethernet interfaces. Old twisted-pair networks operate 
at 10 megabits per second and are called 10Base-T networks. The current standard is 100Base-T (running 
at 100 Mb/s); it's sometimes called Fast Ethernet. 100Base-T requires category 5 cable, and 10Base-T 
devices work with most 100Base-T switches and hubs. If you need something a little bit faster, you can go 
to Gigabit Ethernet; 1000Base-T uses twisted-pair like its predecessors, and 1000Base-SxX is the fiber- 
optic version. If you don't run a supercomputer center, it's unlikely that you'll need anything like this — 
you'll be hard-pressed to saturate a 100Base-T network. 


To build a network, you need a network interface card (NIC) for each of your computers, a hub or switch, 
and some cables to connect the computers to the hub or switch. If you run out of network ports, it's easy to 
add another hub or switch. 


Each node on an Ethernet network has a hardware address, also called the MAC (Media Access Control) 
address. Running ifconfig -a shows yours; in the example in Section 5.4, the hardware address was 
00 :40:05:A0:7F:96. 


The first three bytes of a MAC address make up avendor code, or OUI (Organizationally Unique 
Identifier). For example, Sun Microsystems’ OUI is 08-00-20. You can look up an OUI at 

http://standards. ieee.org/regauth/oui/index.shtml. You do not need to bother with the MAC address except 
when debugging. 


Note Each node on your network should have a different MAC address even if two Ethernet cards are 
of the same model, and most Ethernet devices carry a label with the MAC address. 
Unfortunately, you may come across a batch of NIC cards with a duplicate address or two. You 
could try to return the duplicates, but many modern cards also allow you to change the MAC 
address with an extra utility. 


5.10.1Ethernet and IP 


IP over Ethernet is very straightforward. When you want to send a packet to some other host on your local 
area network, your machine must know that host's MAC address. Using a series of ARP (Address 
Resolution Protocol) broadcasts, your host maps the IP address of the target to the target's MAC address 
and then sends the IP packet out on the wire to that MAC address. 


Your host also caches the destination MAC address inside the kernel. To view your system's current ARP 
cache, run the following command: 


arp -a 


If you don't have DNS working yet, use arp -na instead, to disable hostname resolution. Linux arp with 
no arguments gives a slightly more formatted list than either of the above options, but this format is not 
standard across all versions of Unix. 


Note Remember that ARP only applies to machines on local subnets. To reach destinations outside 
your subnet, your host sends the packet to the router, and it's someone else's problem after that. 


The only real problem you can have with ARP is that your system's cache can get out of date if you're 
moving an IP address from one network interface card to another, because the cards have different MAC 
addresses (for example, this can happen when testing a machine). Unix systems invalidate the ARP cache 
entry if there is no response after a while, so there shouldn't be any trouble other than a small delay for 
invalidated data. However, if you want to delete an ARP cache entry immediately, use this command: 


arp -d host 


You can also view the ARP cache for a single network interface like this: 


arp -i interface 


The arp(8) manual page explains how to manually set ARP cache entries, but you should not need to do 
this. 


Note Don't confuse ARP with RARP (Reverse Address Resolution Protocol). RARP transforms a MAC 
address back to a hostname or IP address. Before DHCP became popular, some diskless 
workstations and other devices used RARP to get their configuration, but RARP is rare today. 


5.10.2Private Networks 


Let's say that you decide to build your own network at home or at the office. You have your Ethernet cards, 
hub or switch, and cables ready. Your next question is, "What IP subnet should | use?" 


You can pay your ISP for a block of Internet addresses if you need real Internet routing to a number of 

individual hosts inside your local network. However, this costs a lot of money and isn't useful for anything 
but a highly decentralized site, such as a university. As a much less expensive alternative, you can pick a 
subnet from the addresses in the RFC 1918 Internet standards document, shown in Table 5-2 on the next 


page. 
Table 5-2: Private Networks Defined by RFC 1918 


Network Subnet Mask Short Form 


10.0.0.0 | 255.0.0.0 | 10.0.0.0/8 


192.168.0.0 255.255.0.0 192.168.0.0/16 
172.16.0.0 255.240.0.0 172.16.0.0/12 


Unless you plan to have more than 254 hosts on a single network, pick a small subnet such as 
10.0.0.0/255.255.255.0, which uses the addresses 10.0.0.1 through 10.0.0.254. (Networks with this 
netmask are sometimes called class C subnets. This term is technically somewhat obsolete but still useful 
in practice.) You can carve up private subnets in any way that you please, experimenting to your heart's 
content. 















So what's the catch, you say? It's very simple — hosts on the real Internet know nothing about these 
private subnets and will not route to them. With no extra help, your hosts cannot talk to the outside world. 
Therefore, if you have a single Internet connection, you need to have some way to fill in the gap between 
that connection and the rest of the hosts on your private network. Network Address Translation (NAT) does 
exactly this; see Section 5.14. Refer back to Figure 5-1 on page 94 for an illustration of a typical network. 
The gateway can be a regular Linux machine or one of the devices described in Section 5.9.1. 


5.11Configuring Routes 


Routing is the act of transferring packets from one host or subnet to another. Let's say that you have two 
LAN subnets, 10.0.0.0/24 and 10.0.1.0/24, and a Linux router machine with two Ethernet cards, one 
connected to each subnet. The router has two IP addresses: 10.0.0.1 for etho and 10.0.1.1 for eth1. 
Figure 5-4 shows the two networks; the router's routing table looks like this (obtained by running route - 


n): 

Destination Gateway Genmask Flags Metric Ref Use Iface 
TOs Oe O20 2060.0 BOTs a nO U O O O eth0 
1000 020.020 Nelo wn 4s ow tele D U O O O eth1 





Figure 5-4: Two subnets joined with a router. 


Furthermore, let's say that the hosts on each subnet have the router as their default gateway (10.0.0.1 for 
10.0.0.0/24 and 10.0.1.1 for 10.0.0.0/24). Therefore, if 10.0.0.37 wanted to send a packet to anything 
outside of 10.0.0.0/24, it would pass the packet to 10.0.0.1. Now let's say that you want to send a packet 
from 10.0.0.37 to 10.0.1.23. The packet goes to 10.0.0.1 (the router) via its et ho interface, and now you 
want it to go back out through the router's eth1 interface. To make the Linux kernel perform this basic 
routing function, the only thing you need to do is enable /P forwarding on the router with the following 
command: 


echo 1 > /proc/sys/net/ipv4/ip_forward 


This is easy enough, but what if you have another subnet, 10.0.2.0/24, connected to the host at 10.0.0.37 
on that host's second network interface, as shown in Figure 5-5? After configuring 10.0.0.37's second 
Ethernet interface to 10.0.2.1, you now need to figure out how everything else in 10.0.0.0/24 and 
10.0.1.0/24 can talk to 10.0.2.0/24. Let's start with the router that connects 10.0.0.0/24 and 10.0.1.0/24. 


16,0.0,4 10.0,0.37 10.0.0. 731 


10.0.0.1 | etho 
( 10.0.2.0/24 — | Router | 
10.0.1.1 | eth! 


Host 


16.0, 1.40 10.0. 1.67 


Figure 5-5: Three subnets. 


You can tell the router that 10.0.0.37 handles 10.0.2.0/24 with this command: 


route add -net 10.0.2.0' netmask 255,255.2559.0 gw 10.0.0437 


The routing table on the router now looks like this: 


Destination Gateway Genmask Flags Metric Ref Use Iface 
Toe Usa dO hee eee 1 PODS a e ge Owe UG O O O eth0 
TOU Us Oels AEA e w U O O O eth0 
Weges UUs Us U pase P A a E o O U O O O eth1 


As an added bonus, recall that all traffic from 10.0.1.0/24 initially goes to the router, because 10.0.1.1 is 
the default router on that subnet. Therefore, anything on 10.0.1.0/24 can now talk to 10.0.2.0/24, and vice 
versa (as long as you set the default route for the hosts on 10.0.2.0/24 to 10.0.2.1). But what about 
10.0.0.0/24? 


Technically, this also works now, because the packets go to 10.0.0.1 (eth0 on the router), then back out 
the same network interface to 10.0.0.37. This is inefficient and a bit slower, of course, because the packets 
to 10.0.2.0/24 must go across the same wire twice, with the router handling the packet between the 
transmissions. If you want to "fix" this, you must run a route command similar to the one above for each 
host on 10.0.0.0/24. 


Say that the router has a connection to the Internet, and that this is the router's default gateway. 
Theoretically, there's no problem in sending packets out of your network to the rest of the Internet. 
Unfortunately, if your IP addresses are in private networks (as in this section) you run into the same 
problem described in Section 5.10.2 — you will never get anything back. Again, you need to run NAT (see 
Section 5.14) or do some other trick to get everything within the network talking to the outside world. 


5.12The Transport Layer: TCP, UDP, and Services 


So far, you have only seen how packets move from host to host on the Internet. That's clear enough, but it 
says nothing about how your computer presents the packet data it receives to its running processes. You 
want more than one application to be able to talk to the network at the same time (for example, you might 
have email and several Web clients running), and you also want to receive the data in some form that's 
easier to handle than packets. 


Transport layer protocols and services perform these tasks. The two most popular transport protocols are 
the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP). Both define ports on 
network nodes. When a program on your machine wants to talk to a service on a remote machine, it opens 
a port on the local machine and requests a connection to a port on the remote machine (a connection is 
defined by the two hosts, the ports on each side, and a special packet sequence). After the transport layer 
services on both sides establish the connection, the hosts can exchange data. 


Transport layer services have complex implementations because they must convert a series of packets 
(not necessarily arriving in the correct order) into a data stream for programs. Luckily, you need to know 
next to nothing about this mess. 


In TCP and UDP, a port is a number. If a program accepts connections on a port, it is said to listen on that 
port. There are well-known ports for various services; for example, email servers listen on TCP port 25, 
and most Web servers listen on TCP port 80. Only the superuser may use ports 1 through 1023. All users 
may listen on and create connections from ports 1024 on up. 


Ona Unix system, the /etc/services file maps service names to port numbers and transport layer 
protocols. This plain-text file consists of lines like these: 


ssh ZAP CD # SSH Remote Login Protocol 
smtp Z25/tCp 
domain 52/ udp 


The first column holds the service names. The second column in the /etc/services file defines the port 
number and a transport layer protocol. Take a look at /etc/services on your machine. Notice that the 
services file does not define the programs that listen on ports on your computer. 


You will learn more about how programs listen and connect to the application layer in the next chapter. 
The rest of this chapter is devoted to three advanced networking topics: firewalls, NAT (IP masquerading), 
and wireless networking. 


5.13Firewalls 


Afirewallis a machine that sits between a network and the rest of the Internet, attempting to ensure that 
nothing "bad" from the Internet harms the network. You can also set up firewall features for each machine, 
where the machine screens all of its incoming and outgoing data at the packet level (as opposed to the 
application layer, where server programs usually try to perform some access control of their own). 
Firewalling on individual machines is sometimes called /P filtering. 


To understand how firewalls work, consider that there are three times when a system can filter packets: 
m When the system receives a packet 
m When the system sends a packet 
=» When the system forwards (routes) a packet to another host or gateway 

With no firewalling in place, a system just processes packets and sends them on their way. 


Firewalls put checkpoints for packets at the points of data transfer identified above. The checkpoint drops, 
rejects, or accepts packets, usually based on some of these criteria: 


=» The source or destination IP address or subnet 

=» The source or destination port (in the transport layer information) 

a The firewall's network interface 
In Linux, you create firewall rules in a series known as a chain that is part of a networking table (for the 
most part, you will only work with one table named filter that controls basic packet flow). The 
iptables command, available since kernel version 2.4.0, manipulates these rules and chains. To use 
iptables, your kernel configuration must have packet filtering (in the networking options) and IP tables 


support (in the netfilter configuration submenu). You will learn more about kernel configuration in Chapter 
10. 


There are three basic chains: INPUT for incoming packets, OUTPUT for outgoing packets, and 
FORWARD for routed packets. To view the current configuration, run this command: 


iptables -L 


Theiptables command should list an empty set of chains, as follows: 


Chain INPUT (policy ACCEPT) 
target prot opt source destination 


Chain FORWARD (policy ACCEPT) 
target prot opt source destination 


Chain OUTPUT (policy ACCEPT) 
target prot opt source destination 


Each chain has a default policy that specifies what to do with a packet if no rule matches the packet. The 
policy for all three chains in this example is ACCEPT, meaning that the kernel allows the packet to 
successfully pass through the packet-filtering system. For the purposes of this book, the only other value 
that makes any sense is DROP, which means that the kernel will discard the packet. To set the policy ona 
chain, use iptables -P as in this example: 


iptables -P FORWARD DROP 


Warning Don't do anything rash with the policies on your machine until you've read through the rest of 
this section. 


Let's say that someone at 192.168.34.63 is annoying you, so you'd like to prevent them from talking to 


your machine. Run this command: 


tprables =A INPUT =o 192.163.234.638: ~]: DROP 


The-A INPUT parameter appends a rule to the INPUT chain. The -s 192.168.34.63 part specifies the 
source IP address in the rule, and -j DROP tells the kernel to discard any packet matching the rule. 
Therefore, your machine will throw out any packet coming from 192.168.34.63. 


To see the rule in place, run iptables -L: 


Chain INPUT (policy ACCEPT) 
target prot opt source destination 
DROP all == 2922168 .34 0963 anywhere 


As luck would have it, though, your good buddy at 192.168.34.63 has told everyone on his subnet to open 
connections to your SMTP port (TCP port 25), so now you want to get rid of that traffic as well: 


iptables. =A INPUT =s: 192::168.34.0724 =p tcp ==destination=pore 25.=] DROP 


This example adds a netmask qualifier to the source address, as well as -p tcp to specify TCP packets 
only. A further restriction, --destination-port 25 says that the rule should only apply to traffic to port 
25. The IP table list for INPUT now looks like this: 


Chain INPUT (policy ACCEPT) 


target prot opt source destination 
DROP abt => .[92.168,34.:63 anywhere 
DROP ep = «97 166 738470724 anywhere BCD ei SEEE NE 


All is well until you hear from someone you know at 192.168.34.37, saying that they can't send you email 
because you blocked their machine. Thinking that this is a quick fix, you run this command: 


iptables =A INPUT >98. 192.168.342.537 =T ACCEPT 


However, it doesn't work. For the reason, look at the new chain: 


Chain INPUT (policy ACCEPT) 


target prot opt source destination 

DROP alL, ==" LOZ 68.4 So 0S anywhere 

DROP tep ==. 1926634. 0774 anywhere ECO dpt smtp 
ACCEL alk =e TILLO aa ba anywhere 


The kernel reads the chain from top to bottom, using the first rule that matches. The first rule does not 
match 192.168.34.37, but the second does, because it applies to all hosts from 192.168.34.1 to 
192.168.34.254, and this second rule says to drop packets. Once a rule matches, the kernel carries out the 
action and does not look further down in the chain. (You might remark that 192.168.34.37 can send 
packets to any port on your machine except port 25, because the second rule only applies for port 25.) 


Therefore, you have to move the third rule to the top. The easiest way to do this is to first delete the third 
rule with this command: 


iptables -D INPUT 3 


Then, insert that rule at the top of the chain with iptables -I: 


iptables =I INPUT =s. 297.168.534.357 <=) ACCEPT 


If you want to insert a rule at some other place than the top of a chain, put the rule number after the chain 


name (for example, iptables -I INPUT 4 ...). 


Although this little tutorial may show you how to insert rules and how the kernel processes IP chains, it 
doesn't even come close to illustrating firewall strategies that actually work. Let's talk about that now. 


5.13.1Firewall Strategies 


There are two basic kinds of firewall scenarios: one for protecting individual machines (where you set rules 
in each machine's INPUT chain), and one for protecting a network of machines (where you set rules in the 
router's FORWARD chain). In both cases, you cannot expect to have serious security by using a default 
policy of ACCEPT, continuously inserting rules to drop packets from sources that start to send bad stuff to 
you. Instead, you must adopt the strategy of allowing only the packets that you trust and denying 
everything else. 


For example, let's say that your machine has an email server on TCP port 25 and an SSH server on TCP 
port 22. There is no reason for any random host to talk to any other port on your machine, and you 
shouldn't give any such host a chance. Here's how to set it up. 


First, set the INPUT chain policy to DROP: 


iptables -P INPUT DROP 


If you want to enable ICMP traffic (for ping and other utilities), use this line: 


iptables -A INPUT -p icmp -jJ ACCEPT 


Let's say that your host's IP address is my_addr. Make sure that you can receive packets sent from 
yourself to both your own network IP address and 127.0.0.1 (localhost): 


iptables -A INPUT -s 127.0.0.1 =J] ACCEPT 
iptables -A INPUT -s my_addr -jJj ACCEPT 


If you have control of your entire subnet (and trust everything on that subnet), you can replace my_addr 
with your subnet address and subnet mask; for example, 10.43.201.0/24. 


Now, although you need to make sure that your host can make TCP connections to the outside world, you 
still want to deny all incoming TCP connections. Because all TCP connections start witha SYN 
(connection request) packet, if you let all TCP packets through that aren't SYN packets, you are still okay: 


iptables -A INPUT -p tcp "I" -—=-6yn =J ACCEPT 


Next, you must accept traffic from your nameserver so that your machine can look up names with DNS. 
Do this for all DNS servers in /etc/resolv.conf. Assuming that the nameserver's address is ns_addr, 


you can do this with this command: 


iptables ~A INPUT -p udo —-sourte-port 33: -s ns adar =]. ACCEPT 


As mentioned before, you want to allow SSH and SMTP connections from anywhere: 


iptables -A INPUT -p tep —=cestinalion—port 22 =9] ACCEPT # ssh 
iptables -A INPUT =p tcp --destination-port 25 =] ACCEPT # smtp 


Finally, you need the following command to enable outgoing passive FTP connections, though it's not 
terribly critical: 


iptables -A INPUT -p top —=source—-port 20 —-destination—-port. 1024: =] ACCEPT 


The preceding IP table settings work for many situations. For example, they work for any direct 
connection, especially broadband, where an intruder is much more likely to port-scan your machine. They 
work for university environments, which are very similar to regular broadband connections. And you could 
also adapt these settings for a firewalling router, using the FORWARD chain instead of INPUT, and using 
source and destination subnets where appropriate. However, if you are using a firewalling router, you 
might find the discussion of NAT in Section 5.14 more applicable to your situation. 


Remember that the key idea is permitting only the things that you find acceptable, not trying to find stuff 
that is unacceptable. Furthermore, IP firewalling is only one piece of the security picture. See Section 6.7 
for more information. 


5.13.2IP Tables Reference 


This section is a quick reference for the most common iptables options. 


Alliptables invocations must have a command option, as listed in Table 5-3. 


Table 5-3: iptables Command Options 





Command Option | Description 








-Lchain Lists rules (chain is optional). 

-Achain  Appends a rule to chain. 

-Ichain Inserts a rule in chain (rulenum is optional; the default location is the 
rulenum beginning of the chain). 

-Dchain Deletesrulenum from chain. You can omit the rule number if you specify 
rulenum a matching parameter (shown in Table 5-4). 

-Fchain Deletes all rules in chain, leaving only the policy. 

rulenum 

-Pchain Sets the policy (default rule) of chain to newpolicy. 





newpolicy 





Table 5-4: iptables Packet Specification Options 





Parameter Option Meaning 

-saddr/mask Selects source IP address addr with optional netmask mask (for an 
entire subnet). 

-daddr/mask Selects destination IP address addr with optional netmask mask (for 
an entire subnet). 

—-iinterface Selects input interface (e.g., eth0,ppp0). 

—-ointerface Selects output interface (e.g., eth0,pppo0). 

-pprotocol Selectsprotocol (e.g., tcp,udp,icmp). 

—-source-port Selects source port. You may specify a range with port1l:port2. 

port This works in conjunction with -p tcp or-p udp. 

—-destination- Selects destination port. You may specify a range with 

portport port1:port2. This works in conjunction with -p tcp or-p udp. 











—-syn When using -p tcp, selects SYN (connection request) packets. 





When using -A and -1 (and -D without a rule number), you must specify the rule to add or delete as a 
combination of the packet specification options in Table 5-4. In addition, you can insert a literal ! after the 
option flag to negate its meaning. 


To finish off your rule, use —j target to decide what to do if a packet matches the rule (the most 
common types of target are ACCEPT and DROP). 


This section has only described the basics of iptables. There are many advanced features, such as the 
ability to send a packet to an entirely different destination, to intercept packets at different parts of the 


routing process, and to create your own subchains. You can read about them in the iptables(8) manual 
page. 


5.14Network Address Translation (IP Masquerading) 


Network Address Translation (NAT) permits sharing a single IP address (for example, from a PPP or 
broadband connection) with an entire internal network. NAT is very popular with home and small office 
networks. IPv4 addresses are in short supply, and furthermore, ISPs do not really want to bother with more 
routing than they can handle. As mentioned in Section 5.9.1, many small routers support NAT. In Linux, 
the variant of NAT that most people use is known as /P masquerading. 


The basic idea behind NAT is that the machine with the Internet connection acts as a proxy between the 
internal network and the rest of the Internet. Consider a network like the one back in Figure 5-1 on page 
94, and assume that the router (gateway) has an eth0O interface to an internal private network and a pppod 
interface for the uplink to the Internet. 


Every host on the Internet Knows how to connect to the router, but they know nothing about the internal 
private network behind the router. Under NAT, each of the internal hosts has the router as its default 
gateway. The system works roughly as follows: 


1. A host on the internal private network wishes to make a connection to the outside world, so it sends 
its connection request packets through the router as normal. 


2. The router intercepts the connection request packet rather than passing it out to the Internet (where 
it would get lost, because the public Internet Knows nothing about private networks such as 
10.1.2.0/24). 


3. The router determines the destination of the connection request packet and opens its own 
connection to the destination. 


4. When the router obtains the connection, it fakes a "connection established" message back to the 
original internal host. 


5. The router is now the middleman between the internal host and the destination. The destination 
knows nothing about the internal host; the connection on the remote host looks like it came from the 
router. 


This doesn't sound too bad until you think about how it works. Plain old IP routing knows only source and 
destination IP addresses in the Internet layer. However, if the router dealt only with the Internet layer, each 
host on the internal network could establish only one connection to a single destination at one time (among 
other limitations), because there is no information in the Internet layer part of a packet that could 
distinguish multiple requests from the same host to the same destination. 


Therefore, NAT must dissect packets to pull out more identifying information. The only suitable stuff is in 
the transport layer — in particular, the TCP and UDP port numbers. The transport layer is very complex, 
so you Can imagine that there are lots of things that can go wrong when a router goes mucking about 
inside transport layer information that comes from two hosts (the internal one and the external one) that it 
knows nothing about. 


That said, NAT works better in practice than this brief description might suggest. To set up a Linux 
machine to perform as a NAT router, you must activate all of the following inside the kernel configuration: 
network packet filtering ("firewall support"), connection tracking, IP tables support, full NAT, and 
MASQUERADE target support. 


Note SeeChapter 10 for information on the kernel. 


Then you need to run iptables commands like the following to make the router perform NAT for its 
private subnet. The following example commands are for the earlier example: 


echo 1 > /proc/sys/net/ipv4/ip_forward 

iptables -P FORWARD DROP 

iptables -t nat -A POSTROUTING -o pppO -j MASQUERADE 

iptables -A FORWARD -i ethO -j ACCEPT 

iptables -A FORWARD -m state --state ESTABLISHED, RELATED -J ACCEPT 


This example applies to an internal Ethernet network on etho sharing a PPP Internet connection (ppp0). 
Change these to suit your network. 


When using NAT, remember that it's essentially a hack. In a perfect world, we would all be using IPv6 (the 
next-generation Internet), and we could get permanent subnets from our ISPs without any pain. In the 
mean-time, though, you have to deal with these NAT limitations: 


a Internet hosts cannot connect to services inside the private network without special port forwarding 
help from the NAT router. Of course, the hosts in the private network may have network security 
vulnerabilities, so this limitation may be more of a blessing than a drawback. 


=» Because Internet hosts cannot connect to internal hosts, some services need even more complicated 
packet dissection to work. 


=» NAT works for TCP, UDP, and some ICMP, but not necessarily all transport layer protocols. 


a NAT requires much more memory and computing power than normal IP routing, so the computer 
acting as the router cannot be completely underpowered. In addition, there is always some connection 
overhead and latency. However, modern computers are more than fast enough to handle this, and the 
overhead and latency don't really matter unless you're running a big operation. 


5.15Wireless Ethernet 


The proliferation of notebook computers and other portable computing devices has given rise to wireless 
Ethernet (also irritatingly known as "Wi-Fi") that you do not need to physically plug into a switch or hub. 
There are two popular consumer versions of the standard: 802.11b (maximum speed 11 Mbps) and 
802.11g (54 Mbps). 


In principle, wireless Ethernet isn't much different than any other kind of Ethernet, except that it uses radio 
waves instead of copper wires or fiber. You can configure a wireless card's network interface with 
traditional tools, such as ifconfig and dhclient. And wireless Ethernet cards have MAC addresses just 
like their wired counterparts. 


With a wireless network, you typically want to be able to send packets out to some wired network. To 
accomplish this, you can buy a wireless base station to bridge (or link) a wireless network to a wired 
network — it's almost like adding more ports to a hub or switch by plugging another hub or switch into one 
of the ports. A wireless base station contains both wired and wireless networking hardware; it physically 
plugs into the wired network and can also communicate wirelessly with wireless network devices. (You can 
make a regular computer do the work of a wireless base station, but it's hardly worth the trouble.) 


Linux has an additional tool called iwconfig for setting up a few technical details of a wireless network. 
For example, if someone else has a wireless network next door, you need to differentiate yours from theirs 
with an Extended Service Set Identifier (ESSID) string identifier. 


To enable iwconfig, your kernel needs the wireless LAN extensions that you'll find in the kernel 
configuration's networking support options (see Chapter 10). Running iwconfig with no options yields 
output like this: 


lo no wireless extensions. 
etho no wireless extensions. 
ethl IEEE 802.11-DS ESSID:"some net" Nickname: "HERMES I" 


Mode:Managed Fregquency:2.437GHz Access Point: 00:04:5A:CE:BF:17 
Bit Rate:11Mb/s Tx-Power=15 dBm Sensitivity:1/3 

RIS LAr sort Fragment thr:off 

Power Management:off 

Link Quality:44/92 Signal level:-49 dBm Noise level:-93 dBm 

RX invalid nwid:0 invalid crypt invalid mis¢e:7255 


As you can see, wireless settings are attached to network interfaces. The most important settings are as 
follows: 


ESSID A string identifier for the network. Your wireless access point should have this setting. 


Mode A setting that defines the type of network and the network interface's role in the network. Most 
wireless networks are managed networks, meaning that there is at least one interface that serves as 
an access point and oversees the nodes on the network. Your interface can act as a client, the master 
access point, a secondary access point, or a repeater. However, there are also ad hoc networks that 
have no access points. 


Access point The MAC address of your current wireless access point's wireless interface (if you're on 
a managed network). 


Encryption key The wireless network card's current Wired Equivalent Privacy (WEP) key. 


You can manually set these parameters just as you would with ifconfig: 


iwconfig ethl essid mynet mode managed 


In the preceding example, iwconfig configures the eth1 interface with an ESSID of mynet and a 


network topology of managed. The important iwconfig parameters are listed in Table 5-5. See the 
iwcontig(8) manual page for a complete list of parameters. 


Table 5-5: iwconfig Parameters 





Parameter Value 











essid The ESSID string, or any. 

ap The MAC address of your access point, or any. 

mode Managed,Master,Repeater,Secondary, Or Auto. 

channel A channel number (each channel has a predetermined frequency); run iwlist 
interface channel to geta list of available channels. 

freq A direct frequency specification (for example, 2 .412G for 2.412 GHz). 

key Normally, the hexadecimal value of the WEP key, or off to disable WEP. 





For wireless plug-in PC cards, most distributions support the PC card utilities in /etc/pcmcia, where you 
will find a configuration file called wireless.opts. When you plug a wireless card into your machine, the 
PC card services daemon described in Section 11.6 runs /etc/pcemcia/wireless.wireless runs 
iwconfig based on the parameters in wireless.opts, which is a Bourne shell case statement, making 
decisions primarily based on your wireless card's MAC address. The wireless.opts configuration file 
has many sample entries that you should examine. 


To create an entry in this file for a wireless card with MAC address 00:02:2D:B0:EE:E4, you could use the 
following: 


wo, t00 022D BU thn. EA) 
INFO="My Home Network" 
ESSID="my network" 
MODE="Managed" 
RATE="auto" 
KEY="268B645297" 


vy 


You must place this entry in the case statement in wireless.opts before these lines: 


# NOTE : Remove the following four lines to activate the samples below ... 
P SSeS START SECTION TO REMOVE ----------- 


5.15.1A Lecture on Wireless Security 


Unfortunately, using radio waves as a medium is absurdly insecure. Anyone with any kind of wireless 
Ethernet device, even something as small as a Sharp Zaurus, can sidle up near your network and waltz all 
around your packets. Without any additional protective measures, anyone can get access to your network, 
snoop around your packets, or do something much worse. For example, if your wireless network is 
connected to the Internet, anyone can use your network for a wide variety of very illegal activities. 


There are several ways to make it considerably tougher to access your wireless network, and you'd think 
that most people would use them. Shockingly, the overwhelming majority do not. It's easy to set up a 
completely insecure wireless network, whereas to make the network more secure you need to enter a 
whole bunch of numbers into any computer that wants to use it. Who wants to go through all of that trouble 
when the network works just fine out of the box? Not to mention that it's a continuous hassle, especially 
when one of your friends comes over with a notebook. 


Everyone thinks, "Hey, all of this bad stuff can't happen to me." But it can and will if you choose not to 
have any security. Fortunately, even the weakest kind of wireless security, WEP, drives off virtually all 
would-be intruders. If nothing else, you should enable WEP on your access points and wireless network 
interfaces. Yes, this does mean typing in a bunch of numbers, but do you really want to risk being caught 
in the middle of a legal mess? 


You should know that WEP is not terribly secure. Given five to ten million packets, someone snooping on 
your network can figure out your WEP key with a program such as AirSnort. Of course, sniffing five to ten 


million packets takes a while, but if this concerns you, change your keys every now and then (like, say, 
every three million packets). 


In any case, never transmit any sensitive data across your network without serious encryption like SSL or 


SSH, even if you are using WEP. This is especially true if you do not control your network, and also holds 
for wired networks. 


If you have a good reason to be paranoid about security, WEP isn't going to help you much. You should 
require IPSec or SSH port forwarding for all traffic on your wireless network. Many administrators use a 
virtual private network (VPN) to implement an IPSec-based system. 


Chapter 6: Network Services 


Overview 


Chapter 5 explained all layers of Internet networks except the application layer. Network applications 
(clients and servers) interact with the operating system by using a transport layer protocol and interface, 
such as TCP. This chapter covers basic network servers, including the inetd superserver and SSH 
servers. In addition, you will see some tools that will help you debug the servers. 


Unix network servers come in many forms. A server may listen to a port on its own, or it may use a 
superserver such aS inetd. Servers have no common configuration database, and they vary widely in 
features. Most servers have a configuration file to control their behavior (though with no common format), 
and most use the operating system's syslog service for message logging. 


If you understand the system of TCP and UDP ports described in Section 5.12, you won't run into much 
trouble with network servers. Let's start out by getting an idea of how network clients talk to network 
servers. 


6.1 The Basics of Services 


TCP services are among the easiest to understand because they are simple, uninterrupted data streams. 
For example, you can talk directly to a Web server on TCP port 80 to get an idea of how data comes 
across the connection. Run the following command to connect to a Web server: 


telnet www.nytimes.com 80 


You should get a response like this: 


Trying some address... 
Connected to www.nytimes.com. 
Escape character is '*]'. 


Now type this: 


GET / 


Press ENTER twice. The server should send a bunch of HTML text as a response, and then terminate the 
connection. 


There are two important lessons here: 
=» The remote host has a Web server process listening on TCP port 80. 
=» telnet was the client that initiated the connection. 


Note telnet is a program originally meant to enable logins to remote hosts. Although the non- 
Kerberostelnet remote login server is completely insecure (as you will learn later), the 
telnet Client is useful for debugging remote services. telnet does not work with UDP or any 
other transport layer other than TCP. See Section 6.5.3 for information on net cat, a powerful 
general-purpose network client. 


6.2Stand-Alone Servers 


Stand-alone network servers are no different than any other server daemons on your system (such as 
cron), except that they listen on network ports. In fact, you already saw syslogd in Chapter 4, which 
accepts UDP packets on port 514 if it is started with the -r option. 


Here are some other common network servers that you may find running on your system: 


httpd Web server 

1pd Print server (see Chapter 12) 

postfix, gmail, sendmail Mail servers 

sshd Secure shell daemon (see Section 6.4) 

nfsd, mountd Network filesystem daemons 

smbd, nmbd Windows file-sharing daemons (see Chapier 14) 


portmap Remote procedure call (RPC) portmap service daemon 


6.3 The inetd Daemon 


Implementing stand-alone servers for every single service is somewhat inefficient, because each server 
must know all about the network interface API, have its own port configuration system, and so on. In many 
respects, the API for talking to TCP is not much different than the standard I/O system. TCP servers read 
from network ports and return output back to those ports — this is similar to the behavior of cat with no 
arguments in Section 1.2 is similar. 


Theinetd daemon is a superserver that standardizes network port access and interfaces between regular 
programs and network ports. After you start inetd, it reads the inetd.conf file and then listens on the 
network ports defined in that file, attaching a newly started process to every new incoming connection. 


Each line in inetd. conf looks something like this: 


ident stream tcp nowait nobody /usr/sbin/identd identd -i 


There are seven fields here: 


Service name The service name from /etc/services (see Section 5.12). 
Socket type This is usually st ream for TCP and dgram for UDP. 
Protocol The transport protocol, such as tcp or udp. 


Datagram server behavior For UDP, this is wait or nowait. Services using any other transport 
protocol should use nowait. 


User The username that should run the service. Add . group if you want to set a group. 
Executable The program that inetd should connect to the service. 
Arguments The arguments for the executable in the preceding field. The first argument should be the 


name of the program. 


A# Sign in inetd. conf denotes a comment. To deactivate an inetd service, place a # before its entry 
in/etc/inetd.conf and then run the following command to make inetd re-read its configuration file: 


kill -HUP ‘cat /var/run/inetd.pid 


inetd has several built-in services that you might see at the beginning of your inetd.conf, including 
echo,chargen,discard,daytime, and time. You can use these for testing, but otherwise they aren't 
important. 


6.3.1TCP Wrapper: tcpd, /etc/hosts.allow, /etc/hosts.deny 

Before lower-level firewalls took off in popularity, many administrators used the TCP wrapper library for 
host control over network services. In fact, even if you run an IP firewall, you should still use TCP 
wrappers on any service that does not have its own logging mechanism (and many inetd services do not 


have such capabilities). 


Thet cpd program is the TCP wrapper utility for linking inetd with a server. An administrator can modify 
theinetd. conf file to include tcpa, like this: 


finger stream tcp nowait nobody /usr/sbin/tcpd /usr/sbin/in.fingerd 


When someone makes a connection to the finger port, the following happens: 


1. inetd runs tcpd as the name /usr/sbin/in.fingerd. 
2. tcpd verifies the executable /usr/sbin/in.fingerd. 


3. tcpd consults /etc/hosts.allowand /etc/hosts.deny to see if the remote machine has 
permission to connect. 


4. tcpd logs its decision. 


5. lft cpd decides that the remote host is okay, it runs /usr/sbin/in. fingerd. Otherwise, tcpa 
terminates, dropping the connection. 


Here is an example hosts. deny file: 


ALL: .badguys.example.com 
in.fingerd: nofinger.example.com 
portmap: ALL 


You can interpret this as follows: 
=» Noone in the subdomain .badguys.example.com may connect to any TCP wrapper-enabled program. 
a The host nofinger.example.com may not run in. fingerd. 
=» Noone may access portmap (see Section 6.6 for more information on this service). 


The TCP wrapper library reads /etc/hosts.allow before /etc/hosts.deny. If a rule in 
/etc/hosts.allow matches an incoming connection, the TCP wrapper library allows the connection, 
ignoring /etc/hosts.deny. For example, the following line in /etc/hosts.allow enables portmap 
access to the subdomain goodguys.example.com: 


portmap: .goodguys.example.com 


Note Using domain names in a TCP wrapper configuration has the disadvantage that it uses DNS. 
Forcing DNS lookups can hinder performance. It's also not extremely secure; it's possible to 
affect DNS lookups. 


6.3.2xinetd 


Some Linux distributions come with an enhanced version of inetd named xinetd 
(http://www. xinetd.org/). You can read about the xinetd improvements at the Web site, but to summarize, 
xinetd offers built-in TCP wrapper support, better logging, and extended access control. 


If you currently have inetd, should you run xinetd? If you need its features, sure. However, if you only 
have one local server (for example, FAM for the GNOME desktop), and you firewall the external traffic to 
this server anyway, there are probably better things you can do with your time. xinetd administrators 
should also keep careful track of security advisories, because this package has not seen as widespread 
use as the old inetd. 


xinetd does not read inetd.conf, but if you understand the fields in inetd.conf, you won't have 
trouble using xinetd. If you want an alternative to /etc/xinetd.d, you can also use itox to convert 
yourinetd.conf toa xinetd.conf file that xinetd can read. 


To add a new service to xinetd, create afile in /etc/xinetd.d with the name of the service. For 
example, the /etc/xinetd.d/finger file might look like this: 


service finger 


{ 


socket_type = stream 
provocol = tcp 
wait = no 
user = nobody 
passenv = 


server = /usr/sbin/in.fingerd 


server_args = 


To make xinetd recognize the changes, run kill -USR1 onthe xinetd process ID. 


6.4Secure Shell (SSH) 


The secure shell (SSH) is now the de facto standard for remote logins to other machines. It replaces old, 
insecure programs like telnet and rlogin. In addition to being a good example of a stand-alone server 
(for the purposes of this book, at least), SSH has these features: 


=» Encrypts your password and all other session data, protecting you from snoopers. 


=» Tunnels other network connections, including those from X Window System clients. Tunneling is the 
process of packaging and transporting a network connection using another network connection. The 
advantagesof using SSH to tunnel X Window System connections are that SSH sets up the display 
environment for you and encrypts the X data inside the tunnel. 


= Has clients for almost every operating system. 
a Uses keys for host authentication. 


OpenSSH (http:/Awww.openssh.com/) is a popular free SSH implementation for Unix. The OpenSSH client 
isssh and the server is sshd. OpenSSH uses public key cryptography for authentication and less 

complex ciphers for its session data. SSH does not come without its disadvantages; in particular, you need 
the remote host's public key, and you do not necessarily get it in a Secure way (however, you can check it 
manually if you think you're being spoofed). If you would like to know how cryptography works, get your 
hands on Applied Cryptography [Schneier]. 


There are two main SSH protocol versions: 1 and 2. OpenSSH supports both, with version 2 being the 
default. 


6.4.1Installing OpenSSH 


You can install a precompiled binary version of OpenSSH, such as an . rpn file in a Red Hat distribution, 
or you can get the "portable" source code from the OpenSSH Web site to install from source. If you install 
from source code, you need a version of the SSL (Secure Socket Layer) library, preferably OpenSSL. 


In either case, you need to know your OpenSSH configuration directory — it's usually /etc or 
/usr/local/etc. If you install from source code, you can override the default configuration directory by 
using the --sysconfdir=dir parameter to configure (see Chapter 9 for more information on compiling 
software from source code). 


6.4.2The SSHD Server 


To run sshd, you need a configuration file and host keys in the configuration directory. The configuration 
filename is sshd_config. It's easy to confuse this filename with the client's ssh_config setup file, so 
look out. You shouldn't need to change anything in sshd_config, but it never hurts to check. 


The file consists of kKeyword-value pairs, as shown in this fragment: 


Port 22 

#Protocol 2,1 

#ListenAddress 0.0.0.0 

#ListenAddress 

HostKey /usr/local/etc/ssh_host_key 
HostKey /usr/local/etc/ssh_host_rsa_key 
HostKey /usr/local/etc/ssh_host_dsa_key 


Lines beginning with # are comments. Many comments in your sshd_config may indicate default 
values. The sshd_config(5) manual page contains descriptions of all possible values, but these are the 
most important: 


HostKey file Uses file asa host key (host keys are described shortly). 
SyslogFacilityname Logs messages with syslog facility name. 


LogLevellevel Logs messages with syslog level level. 


PermitRoot Loginvalue Permits the superuser to log in with SSH if value Is set to yes; set 
value to no if you do not want to allow this. 


X11Forwardingvalue Enables X Window System client tunneling if value is set to yes. 


XAuthLocationpath Provides a path for xauth; X11 tunneling does not work without xauth. If 
xauth isn't in /usr/X11R6/bin, set path to the full path-name for xauth. 


Host Keys 


OpenSSH has three different host key sets: one for protocol version 1, and two for protocol 2. Each set 
has a public key (witha .pub file extension) and a private key (with no extension). Do not let anyone see 
your private key, even on your own system. If someone gets your host's private key, SSH provides no 
protection against password snooping. SSH version 1 has RSA keys only, and SSH version 2 has RSA 
and DSA keys. RSA and DSA are public key cryptography algorithms. SSH version 2 provides both 
because there are always debates over which one is better. 


The key filenames are as follows: 
Table 6-1: OpenSSH Key Files 


ssh_host_rsa_key Private RSA key (version 2) 


ssh_host_rsa_key.pub | Public RSA key (version 2) 





ssh_host_dsa_key Private DSA key (version 2) 





ssh_host_dsa_key.pub | Public DSA key (version 2) 


ssh_host_key Private RSA key (version 1) 








ssh_host_key.pub Public RSA key (version 1) 





Normally, you do not need to build the keys, because the OpenSSH installation program should do this for 
you. However, this isn't always a given, and you may need to know how to create keys if you plan to use 
programs like ssh-agent. 


To create SSH protocol version 2 keys, use the ssh—keygen program that comes with OpenSSH: 


ssh-keygen -t rsa -N '' -f ssh_host_rsa_key 
ssh-keygen -t dsa -N '' -f ssh_host_dsa_key 


For the version 1 keys, use this command: 


ssh-keygen -t rsal -N '' -f ssh_host_key 


The SSH server (and clients) also use another key file, ssh_known_hosts, which contains public keys 
from other hosts. If you intend to use host-based authentication, the server's ssh_known_hosts file must 
contain the host keys of all trusted clients. 


Starting the SSH Server 


Runningsshd as root starts the server. You may put this in an init.d script to start at boot time. There is 
also a way to start sshd from inetd, but this is usually not a good idea, because the server occasionally 
needs to generate key files, and this process can take a long time. 


sshd writes its PID to /var/run/sshd.pid, so you can terminate the server at any time with this 
command: 


kill ‘cat /var/run/sshd.pid 


6.4.3The SSH Client 


To log in to aremote host, run this command: 


sshremote_username@host 


You may omit remote_usernameé@ if your local username is the same as on host. You can also run 
pipelines to and from an ssh command. 


The SSH client configuration file is ssh_config, and it should be in the same place as your 
sshd_config file. As with the server configuration file, the client configuration file has key-value pairs, 
but you should not need to change anything in there. 


The most frequent problem with using SSH clients occurs when an SSH version 1 public key in your local 
ssh_known_hosts or .ssh/known_hosts file does not match the key on the remote host. Bad keys 
Cause errors or warnings like this: 


CEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEREEEE 

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ 
CEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE 

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! 

Someone could be eavesdropping on you right now (man-in-the-middle attack) ! 
It is also possible that the RSA host key has just been changed. 

The fingerprint for the RSA key sent by the remote host is 

26 C27 fOr Ud: 0dt49404 205255 65. oat Zat Zit 803100411; 

Please contact your system administrator. 

Add correct host key in /home/user/.ssh/known_hosts to get rid of this message. 
Offending key in /home/user/.ssh/known_hosts:12 

RSA host key for host has changed and you have requested 

strict checking. 

Host key verification failed. 


This usually just means that the remote host's administrator changed the keys, but it never hurts to check 
with the administrator if you don't know. In any case, look at this line of the preceding message: 


Offending key in /home/user/.ssh/known_hosts:12 


This tells you exactly where the bad key is: line 12 of a user's known_hosts file. If you do not suspect foul 
play, just remove the offending line or replace it with the correct public key. 


SSH File Transfer Clients 
OpenSSH comes with replacement programs for rcp and ftp called scp and sftp. 


You can use scp to transfer files from a remote machine to your machine, the other way around, or from 
one host to another. It works much like the cp command. Here are some examples: 


scpuser@host: file 
scpfile user@host:dir 
scpuserl@hosti: file user2@host2:dir 


sftp works much like the command-line ftp client, with get and put commands. The remote host must 
have a sftp-server program, but this shouldn't be a problem if the remote host also uses OpenSSH. 


SSH Clients for Non-Unix Platforms 


There are SSH clients for all popular operating systems. The index at the OpenSSH Web page 
(http://www. openssh.com/) gives a full list, but you may wonder which one to choose for Windows or the 
Mac. PuTTY is a good Windows client that includes a secure file-copy program. MacSSH works well for 
Mac OS 9. Mac OS X is based on Unix and includes OpenSSH. 


6.5Diagnostic Tools 


netstat Is one of the most basic network service debugging tools, telling you what ports are open and 
whether any programs are listening on ports. For example, if you want to view all open TCP ports, run this 
command: 


netstat -t 


Table 6-2 lists the net stat options. 


Table 6-2: netstat Options 


Option Description 





-t Prints TCP port information 
-u Prints UDP port information 
-1 Prints listening ports 

-a Prints every active port 














Disables name lookups (useful if DNS isn't working) 


Being able to list open and listening ports is good, but our good old friend 1sof can go one step further. 


6.5.1lsof 


InSection 4.8.1 you saw how 1sof can track open files, but 1sof can also list the programs currently 
using or listening to ports. For a complete list, run this command: 


lsot —i 


The output should look something like this: 


COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME 


portmap 520 daemon 3u IPv4 Lo UDP *#:sunrpe 

portmap 520 daemon 4u IPv4 LoL TCP *:sunroc (LISTEN) 
inetd 522 root 4u IPv4 188 TCP * discard (LISTEN) 
inetd ae root 5u IPv4 189 UDP *:discard 

inetd 522 root 6u IPv4 190 TCP *:daytime (LISTEN) 
inetd 522 root 7u IPv4 ile UDP *:daytime 

inetd JAZ root Su IPv4 r92 TCP *:time (LISTEN) 
inetd DAA root 9u IPv4 os UDP *:time 

inetd 522 root llu IPv4 195 TCP *:auth (LISTEN) 
sshd ses, root 3u IPv4 696 TCP *:ss5sh (LISTEN) 

X 900 root lu IPv4 TAL TCP *: 6000 (LISTEN) 


If you're looking for one port in particular (that is, if you know that a process is using a particular port and 
you want to know what that process is), use this version of the command: 


lsoft =i. pore 


The full syntax is 


ISO =a. protocolthnhostiporrt 


protocol,@host, and :port are all optional. Specifying any of these parameters filters the 1sof output 
accordingly. As with most other network utilities, host and port can be either names or numbers. 


You can disable host-name resolution with the -n option. Finally, 1sof -P forces numeric port listings. 


Note If you dont have 1sof on your system, you can run netstat -ptogetthe processes 
associated with ports. This is a Linux-specific net stat feature, but 1 sof is still far more 
flexible. 


6.5.2tcodump 


If you need to know what's happening on your network, tcpdump puts your network interface card into 
promiscuous mode and reports on every packet that crosses the wire. 


tcpdump with no arguments produces output resembling the following sample, which includes an ARP 
request and Web connection: 


tcpdump: listening on eth0O 

20:36:25.771304 arp who-has mikado.example.com tell duplex.example.com 
20:36:25.774729 arp reply mikado.example.com is-at 0:2:2d:b:ee:4e 
20:36:25.774796 duplex.example.com.48455 > mikado.example.com.www: S 
3200063165:3200063165(0) win 5840 <mss 1460,sackOK,timestamp 38815804[|tcp]> 
(DF) 

20:36:25.779283 mikado.example.com.www > duplex.example.com.48455: S 
3494716463:3494716463 (0) ack 3200063166 win 5792 <mss 1460,sackOK,timestamp 
4620[|tcp]> (DF) 

20:36:25.779409 duplex.example.com.48455 > mikado.example.com.www: . ack 1 win 
5840 <nop,nop,timestamp 38815805 4620> (DF) 

20:36:25.779787 duplex.example.com.48455 > mikado.example.com.www: P 
1:427(426) ack 1 win 5840 <nop,nop,timestamp 38815805 4620> (DF) 
20:36:25.784012 mikado.example.com.www > duplex.example.com.48455: . ack 427 
win 6432 <nop,nop,timestamp 4620 38815805> (DF) 

20:36:25.845645 mikado.example.com.www > duplex.example.com.48455: P 
1:773(772) ack 427 win 6432 <nop,nop,timestamp 4626 38815805> (DF) 
20:36:25.845732 duplex.example.com.48455 > mikado.example.com.www: . ack 773 
win 6948 <nop,nop,timestamp 38815812 4626> (DF) 


9 packets received by filter 
O packets dropped by kernel 


You can tell tcpdump to be more specific by adding some filtering arguments. You can filter based on 
source and destination hosts, networks, Ethernet addresses, protocols at many different layers in the 
network model, and much more. Among the many packet protocols that t codump recognizes are ARP, 
RARP, ICMP, TCP, UDP, IP, IPv6, AppleTalk, and IPX packets. For example, if you want tcpdump to 
output only TCP packets, run this command: 


tepdump tcp 


If you want to see Web packets and UDP packets, use this command: 


tcpdump udp or port 80 


In the preceding examples, tcp,udp, and port 80 arecalled primitives. The most important primitives 
are in Table 6-3: 


Table 6-3: tcpdump Primitives 





Primitive | Packet Specification 


tcp TCP packets 

udp UDP packets 

portport ‘TCP and/or UDP packets to/from port port 
hosthost Packets to or from host 

netnetwork Packets to or from network 


As you saw in the example, or is an operator. Other operators include and and !; you may use 
parentheses for grouping. If you're going to do any serious work with tcpdump, make sure that you read 
the manual page, especially the section that describes the primitives. 











Note Use good judgment when using tcpdump. The output shown earlier in this section includes only 
packet TCP (transport layer) and IP (Internet layer) header information, but you can also make 
tcpdump print the entire packet contents. Even though many network operators make it far too 
easy to look at their network packets, it doesn't mean that you should. Don't snoop around on 
networks other than the ones you own, unless you happen to be in the espionage business and 
understand the risks of having sensitive wiretapped data. 


If you find that you need to do a lot of packet sniffing, you should probably consider a GUI alternative to 
tcpdump named Ethereal. 


6.5.3Netcat 


If you need more flexibility in connecting to a remote host than a command such aS telnet host port 
allows, use net cat (or nc). Netcat can connect to remote TCP/UDP ports, specify a local port, listen on 
ports, scan ports, redirect standard I/O to and from network connections, and more. 


To open a TCP connection to a port, run this command: 


netcat host port 


netcat does not terminate until the other side of the connection ends the connection. This can confuse 
you if you redirect standard input to net cat. You can end the connection at any time by pressing 
CONT ROL-C. 


To listen on a port, run this command: 


netcat =L =p por: 


Note There are two versions of netcat. The somewhat quirky original has just one executable name, 
nc, with afinal version number of around 1.10. However, there is a newer GNU version using the 
namenetcat, though the current version numbers are lower than the original. The new version 
includes several improvements, not the least of which is a manual page. 


The netcat utility is very specific in its treatment of the network connection; in particular, it does not like to 
exit until the network connection has terminated. If this behavior doesn't suit your application (in particular, 
if you'd like the program and network connection to terminate based on the standard input stream), try the 
sock program instead. 


6.6Remote Procedure Call (RPC) 


RPC stands for remote procedure call. The basic idea is that programs call functions on remote programs 
(identified by program numbers), and the remote programs return a result code or message. 


RPC implementations use transport protocols such as TCP and UDP, and they require a special 
intermediary service to map program numbers to TCP and UDP ports. The server is called portmap, and 
it must be running on any machine that wants to use RPC services. 


If you want to know what RPC services your computer has, run this command: 


recinro ~p Localhos 


RPC is one of those protocols that just doesn't seem to want to die. The unpleasant Network File System 
(NFS) and Network Information Service (NIS) systems use RPC, but they are completely unnecessary on 
stand-alone machines. But whenever you think that you've eliminated all need for portmap, something 
else comes up, such as File Access Monitor (FAM) support in GNOME. 


6.7Network Security 


Because Linux is a very popular Unix flavor on the PC platform, it attracts more unpleasant characters 
who try to break into computer systems. Section 5.13 talked about firewalls, but this isn't really the whole 
story on security. 


Network security attracts extremists. Those who are interested in breaking into systems do so because 
they really like to, and those who come up with elaborate schemes to protect themselves do so because 
they really like to swat away people trying to break into their systems. 


Most people would rather just not have to deal with it, but that attitude and a broadband connection will get 
your system compromised in no time. Fortunately, you don't need to know very much to keep your system 
safe. Here are a few basic rules of thumb: 


Run as few services as possible Intruders can't break into services that don't exist on your system. 
If you know what a service is, and you're not using it at the moment, don't turn it on just in case you 
might want it sometime. 


Block as much as possible with a firewall Unix systems have a number of internal services that 
you may not know about, such as TCP port 6000 for the X Window System server. No other system in 
the world needs to know about these services, and no other system in the world should know about 
them, either. Furthermore, it is very difficult to track and regulate the services on your system, 
because many different kinds of programs listen on ports. You can prevent intruders from discovering 
internal services with effective firewall rules. Refer to Section 5.13.1 for an example firewall. 


Keep on top of the services that you offer to the entire Internet If you run an SSH server or 
Postfix or other services, make sure that you have the ability to stay up to date with the software and 
get appropriate alerts. See Section 6.7.4 for some online resources. 


Don't give an account to anyone who doesn't need an account It's much easier to gain superuser 
access from a local account than it is to break in remotely. In fact, given the huge base of software 
(and the resulting bugs and design flaws) available on most systems, it's almost laughably easy to 
gain superuser access to a system after you get to a shell prompt. Don't assume that your friends 
know how to protect their passwords (or to choose good passwords in the first place). 


Avoid installing dubious binary packages They can contain Trojan horses. 


That's the practical end of protecting yourself. You should know a few basics on why it is important to do 
so. There are three basic kinds of network attacks: 


Full compromise This means getting superuser access (full control) of a machine. An intruder can 
accomplish this by trying a service attack, such as a buffer overflow exploit, or by taking over a poorly 
protected user account, and then trying to exploit a poorly written setuid program. 


Denial-of-service (DoS) attacks These prevent a machine from carrying out its network services, or 
they force a computer to malfunction in some other way without any special access. These attacks are 
harder to prevent, but they are easier to respond to. 


Virus or worm Linux users are mostly immune to email worms and viruses, simply because their 
email clients aren't so stupid as to actually run programs that they get in message attachments, and 
Linux isn't as attractive a target as Windows. However, you can create a Linux virus; it's been done 
before, and as Linux gains popularity, it will happen with increasing frequency. Avoid binary software 
distributions, especially from places that you've never heard of. Some network services have been 
susceptible to worms before, and history is doomed to repeat itself. 


6.7.1Where Linux Distributions Stand 


Not long ago, installing certain Linux distributions on easily accessible networks (such as university 
networks) practically ensured security compromises. These distributions activated every single service by 


default and had no default firewalling. It wasn't just Linux, either — Solaris was remarkably good at leaving 
the door open, not to mention the hundreds of Windows exploits, and so on. 


Linux distributions are better now; they do not activate every possible service, and they tend to come with 
preconfigured firewalls. There is no doubt that the firewalls do most of the work. Programmers delight in 
adding new network services, especially for GUI applications and support, but security often takes a 
backseat. Better authentication (such as the Kerberos system) would help tremendously, but this happens 
to be one of the weakest and most disorganized areas of most applications. 


A firewall isn't necessarily the ideal solution, but it does offer a uniform way to block all network traffic. The 
only thing that ever came close to this in the past was the TCP wrapper system, but that only worked for 
inetd servers and applications that specifically included wrapper support. 


In a perfect world, programmers would write invulnerable code, but while you wait for a perfect world, you 
need to know what to look out for. 


6.7.2Typical Vulnerabilities 


There are two important kinds of vulnerabilities that you need to worry about: direct attacks and clear-text 
password sniffing. Direct attacks just try to take over your machine without being terribly subtle. The most 
common type of direct attack is a buffer overflow exploit, where a careless programmer doesn't check the 
bounds of a buffer array. The attacker fabricates a stack frame inside a huge chunk of data, dumps it to 
the remote server, then hopes that the program overwrites its program data and eventually executes the 
new stack frame. It's a somewhat complicated attack, but easy to replicate. 


On the other hand, clear-text passwords can allow intruders to log in to your machine. From there, they will 
inevitably try to gain superuser access locally (which is much easier than making a remote attack), try to 
use the machine as an intermediary for attacking other hosts, or both. 


Note If you have a service that you need to encrypt, but the service offers no native support, you can 
try Stunnel (htto:/Avww.stunnel.org/), an encryption wrapper package much like TCP wrappers. 
Liket cpd, Stunnel is especially good at wrapping inetd services. 


Some servers are chronic targets of attacks because of poor implementation and design. You should 
deactivate the following services if you ever come across them: 


sendmail | would usually not rail against a specific program, but Sendmail has an exceptionally long 
history of exploitation. There are two very good alternative mail servers, Postfix 
(http://www. postfix.org/) and qmail (http:/Awww.qmail.org/). 


ftpd For whatever reason, all FTP servers seem plagued with vulnerabilities. In addition, most FTP 


servers use clear-text passwords. If you have to move files from one machine to another, consider an 
SSH-based solution or an rsync server. 


telnetd, rlogind, rexecd All of these pass remote session data (including passwords) in clear- 
text form. Avoid them unless you happen to have a Kerberos-enabled version. 


fingerd Intruders can get user lists and other information with the finger service. 


6.7.3Port Scanning 


Listing your open ports and firewall configuration with net stat and iptables is a good start for staying 
on top of unwanted traffic, but you may want to go a step further with Nmap (Network Mapper), a program 
that scans all of the ports on a machine or network of machines, looking for potential vulnerabilities. Nmap 
gives you a view from the outside and eliminates guessing which ports are open. Most intruders use 


Nmap, and you can get it at http://www. insecure. org/. 


Warning If someone else controls the network that you want to Nmap (or run Nmap from), ask that 
person if it's all right that you do so. Network administrators watch for port scans and delight 
in stomping on any machine that runs a scan. 


Just run nmaphost to run a generic scan on a host. Here's an example scan: 


Starting nmap 3.30 (http://www.insecure.org/nmap/) at 2003-08-04 16:25 PDT 


[Interesting ports on host (10.1.aZs2Z)* 
(The 1636 ports scanned but not shown below are in state: closed) 
POrT State Service 


o/ Cp open discard 

3/7 Cep open daytime 

RAEES, open ssh 

S172 ep open time 

1117/top open sunrpce 

Bey gece e open auth 

6000/tcp open > SN 

Nmap run completed -- 1 IP address (1 host up) scanned in 0.594 seconds 


The host in this example probably doesn't have any kind of firewall running, because it has quite a few 
open ports. 


Nmap can do much more — have a look at its manual page and the considerable online resources. 


6.7.4Security Resources 


Here are three good security sites: 


http ://www.sans.orq/ Offers training, services, a free weekly newsletter of the top current 
vulnerabilities, sample security policies, and more. 


http ://www.cert.org/ A place to look for the most severe problems. 


http ://www.insecure.org/ This is the place to go for Nmap and pointers to all sorts of network exploit 
testing tools. It's much more open and specific about exploits than many other sites. 


Chapter 7: Introduction to Shell Scripts 


Overview 


If you can enter commands into the shell, you can write Bourne shell scripts. A script is a series of 
commands written in a file, and the shell reads the commands from the file just as it would if you typed 
them into a terminal. 


As with any program on Unix systems, you need to set the executable bit for the script file, but you must 
also set the read bit. The easiest way to do this is as follows: 


chmod +x Scripc 


Thischmod command allows other users to read and execute script. If you don't want that, use the 
absolute mode 700 instead (and refer back to Section 1.17 for arefresher on permissions). 


7.1Shell Script Basics 


All Bourne shell scripts should start with the following line, indicating that the /bin/sh program should 
execute the commands in the script file: 


#!/bin/sh 


Make sure that no whitespace appears at the beginning of the script file. You can list any commands that 
you want the shell to execute following the #! /bin/sh line. Here is avery simple example: 


#1 Joi ny Shi 
# 


# Print something, then run ls 


echo About to run the ls command. 
ls 


A# character at the beginning of a line indicates that the line is a comment; that is, the shell ignores 
anything on a line after a #. Use comments to explain parts of your scripts that are not easy to understand. 


After creating a script and setting its permissions, you can run the script by placing the script file in one of 
the directories in your command path and then running the script name on the command line. 
Alternatively, you can run ./script if the script is located in your current working directory. 


7.1.1Limitations of Shell Scripts 


The Bourne shell manipulates commands and files with relative ease. You already saw the way the shell 
can redirect output in Section 1.14; this is one of the important elements of shell script programming. 
However, shell scripts are not the be-all and end-all of Unix programming. If you're trying to pick apart 
strings or do arithmetic computations, or if you want functions or complex control structures, you're better 
off using a scripting language like Perl, Python, or awk, or perhaps even a compiled language like C. 


Be aware of your shell script sizes. Bourne shell scripts aren't meant to be big (though you will undoubtedly 
encounter some monstrosities in your time). 


7.2Quoting 


Aliteral is a string that you want the shell to leave alone and pass to the command line. To see where a 
literal is useful, consider how the shell normally expands * to all files and directories in the current 
directory, then passes all those items on to the current command line. If you just want a star (*) to be used 
by a command, as you would for grep and other programs that use regular expressions, you need special 
notation to pass a literal *. 


The easiest way to make the shell leave a string alone is to enclose the entire string in single quotes. 
Here's an example with grep and *: 


grep 'r.*t' /etc/passwd 


All characters located between the two single quotes, including any spaces, make up a single literal. 
Therefore, the following command does not work, because it asks the grep command to search for the 
stringr.*t /etc/passwd from the standard input (because there is only one parameter): 


grep 'r.*t /etc/passwd' 


Double quotes (") work just like single quotes, except that the shell expands any variables that appear 
within double quotes, whereas it does not expand variables located between single quotes. You can see 
the difference by running the following command and then replacing the double quotes with single quotes 
and running it again. 


echo "There is no * in my path: SPATH" 


One tricky part to using literals with the Bourne shell is passing a literal single quote to acommand. One 
way to do this is to place a backslash before the single quote character: 


echo I don\'t like contractions inside shell scripts. 


The backslash and quote must appear outside any pair of single quotes. Astring such as 'don\'t results 
in a syntax error. However, oddly, you can enclose the single quote inside double quotes, as the following 
example shows (the output is identical to that of the preceding command): 


echo "I don't like contractions inside shell scripts." 


7.3Special Variables 


Most shell scripts understand command-line parameters and interact with the commands that they run. To 
take your scripts from simple lists of commands to more flexible programs, you need to know how to use 
the special Bourne shell variables. Using special variables is not much different than using any other shell 
variable (see Section 1.8), but you cannot change the value of certain special variables. 


After reading the next few sections, you will understand why shell scripts accumulate many special 
characters as they are written. If you're trying to understand a shell script and you come across a line that 
looks completely incomprehensible, pick it apart piece by piece. 


7.3.1$1, $2, ... 


$1,$2, and all variables named as positive nonzero integers contain the values of the script parameters, 
or argumenis. 


Let's say the name of the following script is named pshow: 


#!/bin/sh 
echo First argument: $1 
echo Third argument: $3 


Runningpshow one two three produces this output: 


First argument: one 
Third argument: three 


The built-in shell command shift is used with argument variables. This command removes the first 
argument ($1) and advances the rest of the arguments forward — that is, $2 becomes $1,$3 becomes 
$2, and so on. Assume that the name of the following script is shiftex: 


#!/bin/sh 

echo Argument: $1 
shift 

echo Argument: $1 
shitt 

echo Argument: $1 
Shite 


Runshiftex one two three. This output appears: 


Argument: one 
Argument: two 
Argument: three 


7.3 .29# 


Thes# variable holds the number of arguments passed to the script. This variable is especially important 
when running shift in aloop to pick through arguments; when $# is 0, no arguments remain, so $1 is 
empty. (See Section 7.6 for a description of loops.) 


7.3.3$@ 


Thes@ variable represents all of the script's arguments, and It is very useful for passing all of a script's 
arguments to one of the commands inside the script. For example, you will learn in Section 12.6 that the 
arguments to Ghostscript (gs) are complicated. Suppose that you want a shortcut for rasterizing a 
PostScript file at 150 dpi, using the standard output stream, but also leaving the door open for passing 
other options to gs. You could write a script like the following that allows for additional command-line 


options: 


#!/bin/sh 
gs -q -dBATCH -dNOPAUSE -dSAFER -sOutputFile=—- -sDEVICE=pnmraw $@ 


Note If a line in your shell script gets too long for your text editor, you can split it up with a backslash 
(\). For example, you can alter the preceding script as follows: 


#!/bin/sh 
gs -q -dBATCH -dNOPAUSE -dSAFER \ 
-sOutputFile=- -sDEVICE=pnmraw $@ 


A double-quoted $@ ("S@") expands to the parameters separated by spaces. 


7.3 -4$0 


Theso variable holds the name of the script, and it is useful for generating diagnostic messages. Let's say 
that your script needs to report an invalid argument that is stored in the SBADPAR™ variable. You can print 
the diagnostic message with the following line so that the script name appears in the error message: 


echo$0: bad option SBADPARM 


You should send diagnostic error messages to the standard error. Recall from Section 1.14.1 that 2>«1 
redirects the standard error to the standard output. For writing to the standard error, you can reverse the 
process with 1>«2. To do this for the preceding example, use this: 


echo $0: bad option SBADPARM 1>&2 


7.3 .59$ 


Thess variable holds the process ID of the shell. 


7.3.6$? 


Thes? variable holds the exit code of the last command that the shell executed. Exit codes are critical to 
mastering shell scripts, and they're discussed next. 


7.4Exit Codes 


When a Unix program finishes, it leaves an exit code for the parent process that started the program. The 
exit code is a number and is sometimes called an error code. When the exit code is zero (0), this means 
that the program ran without problems. However, if the program has an error, it usually exits with some 
number other than 0. 


The exit code of the last command is held in the $? special variable, so you can check it out for yourself at 
your shell prompt: 


< ds / > J/dev/nūll 

S$ echo $? 

O 

S ls /asdfasdf > /dev/null 

ls: /asdfasdf: No such file or directory 
$ echo $? 

1 


You can see that the successful command returned 0 and the unsuccessful command returned 1. 


If you intend to use the exit code of a command, you must use or store the code immediately after running 
the command. For example, if you run athird echo $? just after the preceding series of commands, the 
result would be 0 because the second of the two echo commands completed successfully. 


When writing shell code that aborts a script abnormally, you should use something like exit 1 to pass an 
exit code of 1 back to whatever parent process ran the script. You don't necessarily have to use 1; for 
example, you may want to use different numbers for different conditions. 


Note A few programs like diff and grep use nonzero exit codes to indicate normal conditions. For 
example,grep returns 1 if it finds something matching its pattern in its input, and 0O if not. In this 
case, the nonzero exit code Is not an error. These commands use other exit codes for errors: 
grep and diff use 2 for real problems. If you think a program is using a nonstandard exit code, 
read its manual page. The exit codes are usually explained in the DIAGNOSTICS section. 


7.5Conditionals 


The Bourne shell has special constructs for conditionals, such as if/then/else and case statements. Here is 
a simple script with an if conditional that checks to see if the script's first argument is hi: 


tI/bin/sh 
if | S1 =i Te then 
echo 'The first argument was "hi"' 


else 
echo =n “The first argument was. not TAHLIT == 1 
echo. Ee was: TTro Tma 

ti 


The words if,then,else, and fi in the preceding script are shell keywords. Everything else is a 
command. This is an extremely important point, because one of the commands is [$1 = "hi" ]. The [ 
character is an actual program on a Unix system; it is not special shell syntax. All Unix systems have a 
command called [ that performs tests for shell script conditionals. This program is also known as test — 
careful examination of [ and test should reveal that they share an inode, or one is a symbolic link to the 
other. 


Now you can see why the exit codes in Section 7.4 are so important, because this is how the whole 
process works: 


1. The shell runs the command after the if keyword and collects the exit code of that command. 


2. Ifthe exit code is 0, the shell executes whatever commands follow the then keyword, stopping 
when it reaches an else or fi keyword. 


3. If the exit code is not 0 and there is an else clause, the shell runs the commands after the else 
keyword. 


4. Theconditional ends at fi. 


Note There is a slight problem with the conditional in the preceding example — avery common 
mistake. $1 could be empty, because the user might not enter a parameter. Without a 
parameter, the test reads [="hi" J, andthe / command aborts with an error. You can fix it by 
enclosing the parameter in quotes in one of the two following ways (both are common): 


if g "31" = fa. J) then 
af 7 X'p" = "pri" 17 then 


It is worth repeating that the stuff following if is always a command. You need a semicolon (; ) after the 
test command — if you skip the semicolon, the shell passes then as a parameter to the test command. (If 
you don't like the semicolon, you can put the then keyword on a separate line.) 


Here is an example that uses grep instead of the [ command: 


#!/bin/sh 
if grep -q daemon /etc/passwd; then 
echo The daemon user is in the passwd file. 
else 
echo There is a big problem. daemon is not in the passwd file. 
fi 


There is also an elif keyword that lets you string if Conditionals together like this: 


#!/bin/sh 
if [ 61 = "hi" J+. then 

echo 'The first argument was "hi"! 
elif [ $2 = "bye" J]; then 


echo 'The second argument was "bye"! 
else 


echo -n "Ihe first argument was not “hi” and the second was not “bye"—— 1 
echo They were TMID ITST end "eZ es 
fi 


Don't get too carried away with elif, because the case construct that you will see in Section 7.5.3 is 
usually more appropriate. 


7.5.1&& and || Logical Constructs 


There are two quick one-line conditional constructs that you may see from time to time: && ("and") and | | 
("or"). 


Thesé & construct works like this: 


commandi && command2 


Here, the shell runs command1, and if the exit code is 0, it also runs command2. 


The| | construct is similar; if the command before a | | returns a nonzero exit code, the shell runs the 
second command. 


&& and | | often find their way into use in if tests. In both cases, the exit code of the last command run 
determines how the shell processes the conditional. In &&, if the first command fails, the shell uses its exit 
code for the if statement, but if the first command is successful, the shell uses the exit code of the 
second command for the conditional. For | |, the shell uses the exit code of the first command if 
successful, or the exit code of the second if the first is unsuccessful. 


Here is an example: 


#!/bin/sh 

1f [ nei” — hi ] | | [ "SJU — bye i- then 
echo 'The first argument was "'S1'™' 

Fi 


If your conditionals include the test ( [) command (as here), you can use -a and -o instead of && and | |, 
as described in the next section. 


7.5.2Testing Conditions 


You have already seen how [ works; the exit code is O if the test is true and nonzero when the test fails. 
You also know how to test string equality with [strl = str2 ]. 


However, remember that shell scripts are well suited to operations on entire files. An important reason for 
this is that the most useful [ tests involve file properties. For example, the following line checks whether 
fileisa regular file (not a directory or special file): 


| =E file | 


In a script, you might see the -f test in a loop similar to this next one, which tests all of the items in the 
current working directory: 


ror filename an *3 do 
if [| -f $filename ]; then 
ls -l Sfilename 
file Sfilename 
else 
echo $filename is not a regular file. 
fi 
done 


You can invert a test by placing the ! operator before the test. For example, [!-ffile ] returns true if 
file is nota regular file. Furthermore, the -a and -o flags are the and and or operators (for example, [- 
ffilel-afile2 }). 


There are dozens of test operations that fall into three general categories: file tests, string tests, and 
arithmetic tests. The info pages contain complete online documentation, but the test(1) manual page is 
also a fast reference. The following sections outline the main tests (some of the less common tests have 
been omitted). 


File Tests 


Most file tests, like —£, are called unary operations because they require only one argument — the file to 
test. 


Here are two important file tests: 
—e Returns true if a file exists 


-s Returns true if a file is not empty 


Several operations inspect a file's type, meaning that they can determine whether something is a regular 
file, a directory, or some kind of special device. Those operations are listed in Table 7-1. There are also a 
number of unary operations that check a file's permissions, as listed in Table 7-2 on the next page (see 
section 1.17 for an overview of permissions). 


Table 7-1: File Type Operators 


Operator Tests For 
-f | Regular file 
_<_ Directoy OOOO 
-h Symbolic link 

=b Block device 
-cC | Character device 

-p Named pipe 

Socket 














Table 7-2: File Permissions Operators 





Operator | Returns True if the File Is: 


-r Readable 

-w Writable 
-x | Executable 

-u Setuid 

= Setgid 

-k "Sticky" 


Note Except for -h,test follows symbolic links. That is, if 1ink is a symbolic link to a regular file, / 
-f link ] returns an exit code of true (0). 











Finally, there are three binary operators (tests that need two files as arguments) that are used in file tests, 
but they are not terribly common. Consider this command that includes -nt (newer than): 


[filel -nt file2 | 


This exits true if file1 has a newer modification date than file2. The -ot (older than) operator does the 
opposite. And if you need to detect identical hard links, -e£ compares two files and returns true if they 
share inode numbers and devices. 


String Tests 


You have already seen the binary string operator =, which returns true if its operands are equal. The != 
operator returns true if its operands are not equal. 


There are two unary string operations: 
-z Returns true if its argument is empty ({[-z "" ] returns 0) 


-n Returns true if its argument is not empty ([-n "" ] returns 1) 


Arithmetic Tests 


It's important to recognize that = looks for string equality, not numeric equality. Therefore, [1=1] returns 
O (true), but [01 = 1 ] returns false. If you want to work with numbers, use -eq instead — [01-eq 1 
] returns true. Table 7-3 provides the full list of numeric comparison operators. 


Table 7-3: Arithmetic Comparison Operators 


Operator | Returns True When the First Argument Is . . . the Second 
-eq Egqaadito 

-ne Not equal to 

=e Less than 

—gt Greater than 


-le | Less than or equal to 


Greater than or equal to 














7.5.3Matching Strings with case 


Thecase keyword forms another conditional construct that is exceptionally useful for matching strings. 
case does not execute any test commands and therefore does not evaluate exit codes. Here is an 
example that should tell most of the story: 


#! /bin/ shi 
case $1 in 
bye) 
echo Fine, bye. 
a 
hi|hello) 
echo Nice to see you. 
aa 
what* ) 
echo Whatever. 
a 
*) 
echo 'Huh?' 
ri 
esac 


The shell executes this as follows: 
1. The script matches $1 against each case value demarcated with the) character. 


2. |f a case value matches $1, the shell executes the commands below the case until encountering 
;;. The shell then skips to the esac keyword. 


3. The conditional ends with esac. 


For each case value, you can match a single string (Such as bye in the preceding example), multiple 
strings with | (hi | hello returns true if $1 equals hi or hello), or make use of the * or ? wildcards 
(what *). If you want to make a case that catches all possible values other than the case values specified, 
use a single *, as shown by the final case in the preceding example. 


Note Each case must end with a double semicolon (; ;). You risk a syntax error otherwise. 


7.6Loops 


There are two kinds of loops in the Bourne shell; for loops and while loops. The for loop is much more 
common; it is actually a "foreach" loop. Here's an example: 


417 Din Ssh 

fOr str in One two three Tour; do 
echo Sstr 

done 


Here, for,in,do, and done are all shell keywords. The shell executes the code as follows: 


1. The shell sets the variable str to the first of the four space-delimited values following the in 
keyword (one). 


2. Theshell runs the echo command between the do and done. 


3. The shell goes back to the for line, setting str to the next value (two), runs the commands 
betweendo and done, and repeats the process until it is through with the values following the in 
keyword. 


Therefore, the output of this script looks like this: 


one 
TWO 
three 
Tour 


The Bourne shell's while loop uses exit codes, like the if conditional. This example does ten iterations: 


#1! /bin/sh 
FILE=/tmp/whiletest.$$§; 
echo firstline > SFILE 


while tail -10 SFILE | grep -q firstline; do 
echo -n Number of lines in SFILE:' ' 
wo =L SFILE | awk *{print Si} 
echo newline >> SFILE 

done 


rm -f SFILE 


Here, the exit code of grep -q firstline Is the test. As soon as the exit code Is nonzero (in this case, 
when the string first line no longer appears in the last ten lines in SF TLE), the loop exits. You can 
break out of a while loop with the break statement. 


The Bourne shell also has an until loop that works just like while, except that it breaks the loop when it 
encounters a zero exit code rather than a nonzero exit code. This said, you shouldn't need to use the 
while and untilloops often. The somewhat obscure example in this section is indicative of the nature 
Ofwhile loops; if you need to use while, you should probably be using a language like awk or Perl 
instead. 


7./Command Substitution 


The Bourne shell can redirect a command's standard output back to the shell's own command line. That is, 
you can use a command's output as an argument to another command, or you can store the command 
output in a shell variable. You do this by enclosing a command in backquotes (`). 


Here's an example that stores a command inside a variable, FLAGS: 


#!/bin/sh 
FLAGS= grep “flags /proc/cpuinfo | sed 's/.*://' | head -1° 
echo Your processor supports: 
for f in SFLAGS; do 
case Sf in 
fpu) MSG="floating point unit" 
v7 
3dnow) MSG="3DNOW graphics extensions" 


TT 


mtrr) MSG="memory type range register" 
rr 
=) MSG="unknown" 
rr 
esac 
echo Sf: SMSG 
done 


The second line contains the command substitution, in bold type. This example is somewhat complicated, 
because it demonstrates that you can use both single quotes and pipelines inside the command 
substitution backquotes. The result of the grep command is sent to the sed command (more about sed 
later), which removes anything matching the expression .*:, and the result of sed is then passed to 
head. 


It's too easy to go overboard with command substitution. For example, don't use `1s` in a script, because 
using the shell to expand * is faster. Also, if you want to invoke a command on several filenames that you 
get as a result of a find command, you may want to consider using a pipeline to xargs rather than 
command substitution (see Section 7.10.4). 


7.8Temporary File Management 


Sometimes it is necessary to create a temporary file to collect output for use by a later command. When 
making such a file, you must make sure that the filename is unique enough that no other programs will 
accidentally write to the temporary file. 


You should know how to use the mktemp command to create temporary filenames. Here is a script that 
shows you what device interrupts have occurred in the last two seconds: 


ly Dany sh 
TMPFILE1= mktemp /tmp/iml.XXXXXX° 
TMPFILE2= mktemp /tmp/im2.XXXXXX° 


cat /proc/interrupts > STMPFILE1 
sleep 2 

cat /proc/interrupts > STMPFILE2 
diff STMPFILE1] STMPFILE2 

rm -f STMPFILE1] STMPFILE2 


The argument to mktemp is a template. mktemp converts the XXXXXX to a unique set of characters and 
creates an empty file with that name (without the Xxxxxx, the command fails). Notice that this script uses 
variable names to store the filenames, so that you only have to change one line if you want to change a 
filename. 


Note Not all Unix flavors come with mkt emp. If you're having portability problems, you might want use 
thess special variable to construct a temporary filename based on the process ID. 


Another problem is that scripts that employ temporary files are vulnerable to signals that abort the script 
and leave temporary files behind. In the preceding example, pressing CONT ROL-C before the second cat 
command leaves a temporary file in /tmp. You want to avoid this if possible. Use the trap command to 
create a signal handler to catch the signal that CONTROL-C generates. The handler removes the 
temporary files: 


7. / bins sh 

TMPFILE1= mktemp /tmp/iml.XXXXXX° 
TMPFILE2="mktemp /tmp/im2.XXXXXX° 

trap “rm -f STMPFILE] STMPFILE2; exit 1" INT 


Notice that you must use exit in the handler to explicitly end script execution. Otherwise, the shell 
continues running as usual after running the signal handler. 


7.9Here Documents 


Sometimes you want to print a large section of text or feed a lot of text to another command. Rather than 
using several echo commands, you can employ the shell's here document feature. 


The following is a script that shows how here documents work: 


#!/bin/sh 
DATE= date 
Ccat<<EOF 
Date: SDATE 


The output above is from Unix date command. 
It's not a very interesting command. 
EOF 


The items in bold control the here document. <<EOF tells the shell to redirect all lines that follow to the 
standard input of the command that precedes <<EOF, which in this case is cat. The redirection stops as 
soon as the EoFmarker occurs on a line by itself. The marker that you use doesn't have to be EOF — use 
any string that you like, but remember that you must use the same marker at the beginning and end of the 
here document, and that convention dictates that the marker be in all uppercase letters. 


Notice also that there is a shell variable in the here document. The shell expands shell variables inside 
here documents. This is especially useful when you're printing out some kind of report and have many 
variables to put into a larger form. 


7.10Important Shell Script Utilities 


There are several programs that are particularly useful in shell scripts. Some utilities (Such as basename) 
are really only practical when used in conjunction with other programs, and therefore usually don't find a 
place outside of shell scripts, but others, such as awk, can be quite useful on the command line too. 


7.10.1basename 


If you need to strip the extension off of a filename or get rid of the directories in a full pathname, use the 
basename command. 


Try these examples on the command line to get a feel for how the command works: 


basename example.html .html 
basename /usr/local/bin/example 


In both cases, basename returns example. The first command strips the .htm1suffix from 
example.html, and the second removes the directories from the full pathname. 


Here is an example of how you can use basename in a script to convert GIF image files to the PNG 
format: 


#!/bin/sh 
for fale in *,glri?7 dö 

# exit if there are no files 

if [| ! -f $file ]; then 

exit 

fi 

b= basename $file .gif` 

echo Converting Sb.g1if to Sb.png.... 

Gitcvopnm, >b.g1f | pnmtopng > Sb.pngd 
done 


7.10.2awk 


You may have seen the awk command already. This is not a simple single-purpose command — it's a 
powerful programming language. Perhaps unfortunately, awk is now something of a lost art due to larger 
languages such as Perl. 


However, entire books do exist on the subject of awk, including The AWK Programming Language [Aho 
1988]. This said, you will see many, many people use awk to pick a single field out of an input stream like 
this: 


ls =1. | awk {print 65}' 


This command prints the fifth field of the 1s output (the file size); the result is a list of file sizes. 


7.10.3sed 


sed stands for stream editor, and it is an automatic text editor that takes an input stream (a file or the 
standard input), alters it according to some expression, and prints the results to standard output. In many 
respects,sed is somewhat similar to ed, the original Unix text editor. It has dozens of operations, 
matching tools, and addressing capabilities. Like awk, there are books about sed, including one that 
covers both utilities, sed & awk [Dougherty]. 


Althoughsed is a big program, and an in-depth analysis is beyond the scope of this book, it's easy to see 
how it works. In general, sed takes an address and an operation as one argument. The address is a set of 
lines, and the command determines what to do with the lines. 


For example, the following command reads /etc/passwd, deletes lines three through six and sends the 
result to the standard output: 


sed3,6d /etc/passwd 


In this example, 3, 6 is the address (a range of lines), and dis the operation (delete). If you omit the 
address,sed operates on all lines in its input stream. The two most common sed operations are probably 
s (search and replace) and d. 


Let's go through a few more sed examples. In all of the examples, single quotes are necessary to prevent 
the shell from expanding special characters like * and $. 


The following command replaces the regular expression exp with text (see Section 1.5.1 for basic 


information on regular expressions) : 


sed 's/exp/text/' 


The preceding command replaces only one instance of the expression exp per line. To replace all 
instances of exp, use the g modifier at the end of the operation: 


sed 's/exp/text/g' 


You can also use a regular expression as the address. The following command deletes any line that 
matches the regular expression exp: 


sed '/exp/d' 


7.10.4xargs 


If you ever have to run one command on a huge number of files, the command or shell sometimes 
responds that it can't fit all of the arguments in its buffer. Use xargs to get around this problem. xargs 
runs a command on each filename in its standard input stream. 


Many people use xargs in conjunction with the find command. Here is an example that can help you 
verify that every file in the current directory tree that ends with . gif is actually inthe GIF format: 


find . -name '*.gif' -print | xargs file 


In the preceding example, xargs runs the file command. However, this invocation can cause errors or 
leave your system open to security problems, because filenames can include spaces and newlines. If 
you're writing a script or need extra security, use the following form instead, which changes the find 
output separator and the xargs argument delimiter from a newline to a NULL character: 


find » -name '*.,gif' =printO | xargs -0 file 


Keep in mind that if you have a large list of files, xargs starts a ljotof processes. Don't expect great 
performance. 


Note You may need to add two hyphens (--) to the end of your xargs command if there is a 
possibility that any of the target files start with -. The —- is a way to tell a program that any 
arguments that follow the —- are filenames, not options. However, keep in mind that not all 
programs support -—-. 


7.10.5expr 


If you need arithmetic operations in your shell scripts, the expr command can help you (and even do 
some string operations). The command expr 1 + 2 prints 3; run expr —-—help for a full list of 
operations.expr is a clumsy, slow way of doing math. If you find yourself using expr frequently, it 


probably means that you should be using a language like Perl, Python, or awk instead of a shell script. 


7.10.6exec 


Theexec command is a built-in shell feature that replaces the current shell process with the program you 
name after exec. This is a feature for saving system resources, but remember that there's no return; once 
you run exec in a shell script, the script and shell running the script are gone, replaced by the new 
command. 


You can test it in a shell window. Try running exec cat. After you press CONTROL-D or CONT ROL-C to 
terminate the cat program, your window disappears because its child process no longer exists. 


7.11Subshells 


Sometimes you need to alter the environment in a shell slightly, but don't want a permanent change. You 
can change and restore a part of the environment (such as the path or working directory) using shell 
variables, but that is a clumsy way to go about things. The easy way around these kinds of problems is to 
use a subshell, an entirely new shell process that you can create just for the purpose of running a 
command or two. The new shell has a copy of the original shell's environment, and when the new shell 
exits, any changes you made to its shell environment disappear, leaving the initial shell to run as normal. 


To use a subshell, put the commands to be executed by the subshell in parentheses. For example, the 
following line executes the command uglyprogramin uglydir, leaving the original shell intact: 


(cd uglydir; uglyprogram) 


Here's another example, showing how to add a component to the path that might cause problems as a 
permanent change: 


(PATH=/usr/confusing:SPATH; uglyprogram) 


Pipes and background processes work with subshells too. The following example uses tar to make an 
archive of the entire directory tree within orig and then unpacks the archive into the new directory 
target, effectively duplicating the files and folders in orig: 


tar er = Ofig | (Cd target; tar xvi =) 


Warning Double-check this sort of command before you run it, because you want to make sure that 
thetarget directory exists and is completely separate from the orig directory. 


7.12Including Other Files in Scripts 


If you need to include another file in your shell script, use the . operator. For example, this runs the 
commands in the file config. sh: 


> CONTIG .6n 


This "include" file syntax does not start a subshell, and it can be useful for a group of scripts that need to 
use a single configuration file. 


7.13Reading User Input 


Theread command reads a line of text from the standard input and stores the text in a variable. For 
example, the following command stores the input in Svar: 


readvar 


This is a built-in shell command. It's useful to a certain extent, and when used in conjunction with other 
shell features not mentioned in this book, you can do some more interesting things, but at a certain point 
you have to ask yourself when enough is enough. 


7.14Too Much? 


The shell is so feature-rich that it's difficult to condense its important elements into a single chapter. If 
you're interested in what else the shell can do, you might want to look at some of the books on shell 
programming, such as Unix Shell Programming [Kochan], or the shell script discussion in The UNIX 
Programming Environment [Kernighan]. 


Above all, remember what shell scripts do best — manipulate files and commands. If you find yourself 
writing something that looks convoluted, especially if it involves string or arithmetic operations, then you 
should probably look to a scripting language like Perl, Python, or awk. 


Chapter 8: Development Tools 


Unix is very popular with programmers not just due to the overwhelming array of tools and environments 
available, but also because the system is exceptionally well documented and transparent. On a Unix 
machine, you don't have to be a programmer to take advantage of development tools, and when working 
with the system, you must know something about programming tools because they play a larger role in 
Unix systems management than in other operating systems. At the very least, you should be able to 
identify development utilities and have some idea of how to run them. 


This chapter packs a lot of information into a small space, but you do not need to master everything here. 
Furthermore, you can easily leave the material and come back later. 


8.1 The C Compiler 


Knowing how to run the C compiler can give you a great deal of insight into the origin of the programs that 
you see on your Linux system. The source code for nearly all Linux utilities, and for many applications on 
Linux systems, is written in the C programming language. C is a compiled language, and C programs 
follow the traditional development process: you write programs, you compile them, and they run. After you 
write a C program, you must compile the source code that you wrote into a binary low-level form that the 
computer understands. 


The C compiler on most Unix systems is named cc (on Linux, this is usually a link to gcc), and C source 
code files end with .c. Take a look at the single, self-contained C source code file called hello. c that 
you can find in The C Programming Language [Kernighan and Ritchie]: 


Pine lide <Staio.h= 


main() { 
pranti("Hello, World.\n™); 
} 


To compile this source code, run this command: 


cc hello.c 


The result is an executable named a. out, which you can run like any other executable on the system. 
However, you should probably give the executable another name (such as hello). To do this, use the 
compiler's—o option: 


cc-o hello hello.c 


For small programs, there isn't much more to compiling than that. You may need to add an extra include 
directory or library (see Sections 8.1.2 and 8.1.3), but let's look at slightly larger programs before getting 
into those topics. 


8.1.1 Multiple Source Files 


Most C programs are too large to reasonably fit inside one single source code file. Mammoth files become 
too disorganized for the programmer, and compilers sometimes even have trouble parsing large files. 
Therefore, developers group components of the source code together, giving each piece its own file. 


To compile the .c files, use the compiler's -c option on each file. Let's say that you have two files, 
main.c and aux.c. The following two commands would do most of the work of building the program: 


cc-c main.c 
cc=—c aux.c 


The preceding two compiler commands compile the two source files into two object files (main.o and 


aux.o). An object file is a nearly complete binary that a processor can almost understand, except that 
there are still a few loose ends. First, the operating system does not know how to run an object file by 
itself, and second, you may need to combine several object files to make a complete program. 


To build a fully functioning executable from one or more object files, you must run the linker — the 1d 
command in Unix. Programmers rarely use 1d on the command line, because the C compiler knows how 
to run the linker program properly. To create an executable called myprog from the two object files above, 
run this command to links them: 


cc -~-o myprog main.o aux.o 


Although you can compile multiple source files by hand, as the preceding example shows, it can be hard to 
keep track of them all during the compiling process when the number of source files multiplies. The make 
system described in Section 8.1.5 is the Unix standard for managing compiles. This system is especially 
important in managing the files described in the next two sections. 


8.1.2Header (Include) Files and Directories 


Cheader files are additional source code files that usually contain type and library function declarations. 
For example, stdio.h is a header file (see the simple program in Section 8.1). 


Unfortunately, a great number of compiler problems crop up with header files. Most glitches occur when 
the compiler can't find header files and libraries. There are even some cases where a programmer forgets 
to include a required header file, so some of the source code may not compile. 


Tracking down the correct include files isn't always easy. Sometimes there are several include files with 
the same names in different directories, and it's not clear which is the correct one. When the compiler can't 
find an include file, the error message looks like this: 


badinclude.c:1: notfound.h: No such file or directory 


This message reports that the compiler cannot not find the not found.h header file that the 
badinclude.c file references. This specific error is a direct result of this directive on line 1 of 
badinclude.c: 


+#include <notfound: i> 


The default include directory in Unix is /usr/include; the compiler always looks there unless you 
explicitly tell it not to (for example, with gcc -nostdinc). However, you can make the compiler look in 
other include directories (most paths that contain header files have include somewhere in the name). 


For example, let's say that you find not found.hin /usr/junk/include. You can make the compiler 
see this directory with the -I option: 


cc -c -I/usr/junk/include badinelude.c 


Now the compiler should no longer stumble on the line of code in bad -include .c that references the 
header file. 


You should also be careful of includes that use double quotes ("") instead of angle brackets (<>), like 
this: 


#include "myheader.h" 


Double quotes mean that the header file is not in a system include directory, but that the compiler should 
otherwise search its include path. It often means that the include file is supposed to be in the same 
directory as the source file. If you encounter a problem with double quotes, you're probably trying to 
compile incomplete source code. 


What Is the C Preprocessor (cpp)? 


TheC preprocessor is a program that the compiler runs on your source code before parsing the actual 
program. The preprocessor rewrites source code into a form that the compiler understands; it's a tool for 
making source code easier to read (and for providing shortcuts). 


Preprocessor commands in the source code are called directives, and they start with the # character. 
There are three basic types of directives: 


Include files An #include directive instructs the preprocessor to include an entire file. Note that the 
compiler's—T flag is actually an option that causes the preprocessor to search a specified directory for 
include files, as you saw in the previous section. 


Macro definitions A line such as #define BLAH something tells the preprocessor to substitute 
something for all occurrences of BLAH in the source code. Convention dictates that macros appear 
in all uppercase, but it should come as no shock that programmers sometimes use macros whose 
names look like functions and variables. (Every now and then, this causes a world of headaches. 
Many programmers make a sport out of abusing the preprocessor.) 


Note that instead of defining macros within your source code, you can also define macros by passing 
parameters to the compiler: -DBLAH=something works like the directive above. 


Conditionals You can mark out certain pieces of code with #ifdef,#if, and #endif. The #ifdef 
MACRO directive checks to see if the preprocessor macro MACRO Is defined, and #ifcondition 
tests to see if condition is non-zero. For both directives, if the condition following the "if statement" 
is false, the preprocessor does not pass any of the program text between the #if and the next 
#endif to the compiler. If you plan to look at any C code, you'd better get used to this. 


An example of a conditional directive follows. When the preprocessor sees the following code, it checks to 
see if the macro DEBUG Is defined, and if so, passes the line containing fprintf() onto the compiler. 
Otherwise, the preprocessor skips this line and continues to process the file after the #endif: 


#ifdef DEBUG 
fprintf(stderr, "This is a debugging message.\n"); 
#endif 


Note The C preprocessor doesn't know anything about C syntax, variables, functions, and other 
elements. It understands only its own macros and directives. 


On Unix, the C preprocessor's name is cpp, but you can also run it with gcc —E. However, you will rarely 
need to run the preprocessor by itself. 


8.1.3Linking with Libraries 


The C compiler does not know enough about your system to create a useful program all by itself. Modern 
systems require libraries to build complete programs. A C library is a collection of common precompiled 
functions that you can build into your program. For example, many executables use the math library 
because it provides trigonometric functions and the like. 


Libraries come into play primarily at link time, when the linker program creates an executable from object 
files. For example, if you have a program that uses the math library, but you forget to tell the compiler to 
link against that library, you'll see errors like this: 


badmath.o(.text+0x28) :undefined reference to 'Ssin' 
badmath.o(.text+0x36) :undefined reference to 'pow' 


The most important parts of these error messages are in bold. When the linker program examined the 
badmath.o object file, it could not find the math functions that appear in bold, and as a consequence, it 
could not create the executable. In this particular case, you might suspect that you forgot the math library 
because the missing functions refer to mathematical operations (sine and exponentiation). 


Note Undefined references do not always mean that you're missing a library. One of the program's 
object files could be missing in the link command. It's usually easy to differentiate between 
library functions and functions in your object files, though. 


To fix this problem, you must first find the math library and then use the compiler's -1 option to link 


against the library. As with include files, libraries are scattered throughout the system (/usr/1lib Is the 
system default location), though most libraries reside in a subdirectory named 1ib. For the preceding 
example, the math library file is libm.a (in /usr/1ib), so the library name is m. Putting it all together, 
you would link the program like this: 


cc -o badmath badmath.o -lm 


You must tell the linker about nonstandard library locations; the parameter for this is —L. Let's say that the 
badmath program requires libcrud.ain /usr/junk/1lib. To compile and create the executable, use a 
command like this: 


cc -0o badmath badmath.o -1m —-L/usr/junk/lib -lcrud 


Note If you want to search a library for a particular function, use the nm command. Be prepared for a 
lot of output. For example, try this: nm /usr/lib/libm.a 


8.1.4Shared Libraries 


A library file ending with .a (Such as libm.a) isa Static library. Sadly, the story on libraries doesn't end 
here. 


When you link a program against a static library, the linker copies machine code from the library file into 
your executable. Therefore, the final executable does not need the original library file to run. 


However, the ever-expanding size of libraries has made static libraries wasteful in terms of disk space and 
memory.Shared libraries counter this problem. When you run a program linked against a shared library, 
the system loads the library's code into the process memory space only when necessary. Furthermore, 
many processes can share the same shared library code in memory. 


Shared libraries have their own costs: difficult management and a somewhat complicated linking 
procedure. However, you can bring shared libraries under control if you know four things: 


= How to list the shared libraries that an executable needs 
» How an executable looks for shared libraries 

=» How to link a program against a shared library 

a What the common shared library pitfalls are 


The following sections tell you how to use and maintain your system's shared libraries. If you're interested 
in how shared libraries work, or if you want to know about linkers in general, you can look at Linkers and 
Loaders [Levine], or at "The Inside Story on Shared Libraries and Dynamic Loading," 
[Beazley/Ward/Cooke]. The Id.so(8) manual page is also worth a read. 


Listing Shared Library Dependencies 


Shared library files usually reside in the same places as static libraries. The two standard library directories 
on a Linux system are /lib and /usr/lib. The /1ib directory should not contain static libraries. 


A shared library has a suffix that contains .so, asin libc-2.3.2.so and libc.so.6. To see what 
shared libraries a program uses, run 1ddprog, where prog Is the executable name. Here is the output of 
ldd /bin/bash: 


libreadline.so.2 => /lib/libreadline.so.2 (0x40019000) 
libncurses.so.3.4 => /lib/libncurses.so.3.4 (0x40045000) 
libdl.so6.2 => /lib/libdl.se.2 (0%4006a000) 

libe.80.6 => Jlib/labe.soe.6 (0x4008d000) 
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) 


Executables alone do not know the locations of their shared libraries; they Know only the names of the 
libraries, and perhaps a little hint. A small program named 1d. so (the runtime dynamic linker/loader) finds 
and loads shared libraries for a program at runtime. The preceding 1dd output shows the library names on 


the left — that's what the executable knows. The right side shows where 1d. so finds the library. 


How lId.so Finds Shared Libraries 


The first place that the dynamic linker should normally look for shared libraries is an executable's 
preconfiguredruntime library search path (if one exists). You will see how to create this path shortly. 


Next, the dynamic linker looks ina system cache, /etc/1d.so.cache, to see if the library isin a 
standard location. This is a fast cache of the names of library files found in directories listed in the cache 
configuration file /etc/1d.so.conf. Each line in this file is a directory that you want to include in the 
cache. The list of directories is usually short, containing something like this: 


/usr/X11R6/1lib 
jusr/lib/libc5—compat 


The standard library directories /lib and /usr/1lib are implicit — you don't need to include them in 
/etc/ld.so.conf. 


If you alter 1d.so.conf or make a change to one of the shared library directories, you must rebuild the 
/etc/1d.so.cache file by hand with the following command: 


Ldconrtig =y 


The-v option provides detailed information on libraries that 1dconfig adds to the cache and any 
changes that it detects. 


There is one more place that 1d.so looks for shared libraries: the environment variable 
LD_LIBRARY_PATH. Before discussing this variable, let's look at the runtime library search path. 


Linking Programs Against Shared Libraries 


Don't get into the habit of adding stuff to /etc/1d.so.conf. You should know what shared libraries are 
in the system cache, and if you put every bizarre little shared library directory into the cache, you risk 
conflicts and an extremely disorganized system. When you compile software that needs an obscure library 
path, give your executable a built-in runtime library search path. 


Let's say that you have a shared library named libweird.so.1in /opt/bizarresoft/lib that you 
need to link myprog against. Link the program as follows: 


cc -o myprog myprog.o —-W1,-rpath=/opt/obscure/lib —-L/opt/obscure/lib -lweird 


The-W1,-rpath option tells the linker to include a following directory into the executable's runtime library 
search path. However, even if you use -W1, -rpath, you still need the -L flag. 


Problems with Shared Libraries 


Shared libraries provide remarkable flexibility, not to mention some really incredible hacks, but it's also 
possible to abuse them to the point where your system is an utter and complete mess. Three particularly 
bad things can happen: 


a Missing libraries 
= Terrible performance 
a» Mismatched libraries 


The number one cause of all shared library problems is an environment variable named 
LD_LIBRARY_PATH. Setting this variable to a colon-delimited set of directory names makes 1d. so search 
the given directories before anything else when looking for a shared library. This is a cheap way to make 
programs work when you move a library around, if you don't have the program's source code, or if you're 
just too lazy to recompile the executables. Unfortunately, you get what you pay for. 


Never set LD_LIBRARY_PATH in shell startup files or when compiling software. When the dynamic runtime 
linker encounters this variable, it must often search through the entire contents of each specified directory 
more times than you would care to know. This causes a big performance hit, but more importantly, you can 


get conflicts and mismatched libraries because the runtime linker looks in these directories for every 
program. 


If you must use LD_LIBRARY_PATH to run some crummy program for which you don't have the source (or 
an application that you'd rather not compile, like Mozilla or some other beast), use a wrapper script. Let's 
say that your executable is /opt/crummy/bin/crummy.bin and it needs some shared libraries in 
/opt/crummy/1lib. Write a wrapper script called crummy that looks like this: 


#!/bin/sh 
LD_LIBRARY_PATH=/opt/crummy/1lib 
export LD_LIBRARY_PATH 

exec /opt/crummy/bin/crummy.bin $@ 


AvoidingLD_lLIBRARY_PATH prevents most shared library problems. But one other significant problem 
that occasionally comes up with developers is that a library's application programming interface (API) may 
change slightly from one minor version to another, breaking installed software. The best solutions here are 
preventive: either use a system like Encap (see Section 9.4) to install shared libraries with -w1,-rpath to 
create a runtime link path, or simply use the static versions of obscure libraries. 


8.1.5Make 


A program with more than one source code file or requiring strange compiler options is too cumbersome to 
compile by hand. This problem has been around for years, and the traditional Unix compile management 
utility that eases these pains is called make. You need to know alittle about make if you're running a Unix 
system, because system utilities sometimes rely on make to operate. However, this chapter is only the tip 
of the iceberg. The classic guide for compiled languages and make is The UNIX Programming 
Environment [Kernighan and Pike]. 


make Is a big system, and there are entire books on the subject (such as Managing Projects with make 
[Oram]), but it's not very difficult to get an idea of how it works. When you see a file named Makefile or 
makefile, you know that you're dealing with make. (Try running make to see if you can build anything.) 


The basic idea behind make is the target, a goal that you want to achieve. A target can be a file (a .o file, 
an executable, and so on) or a label. In addition, some targets depend on other targets — for instance, you 
need a complete set of .o files before you can link your executable. The targets on which another target 
depends are called dependencies. 


To build a target, make follows a rule, such as a rule for how to go from a .c source file to a . o object file. 
make already knows several rules, but you can customize these existing rules and create your own. 


The following is avery simple Makefile that builds a program called myprog from aux.c and main.c: 


OBJS=aux.o main.o 
# object files 


all: mMmyprog 


myprog: $(OBJS) 
$(CC) -0o myprog $(OBJS) 


The first line of the Makefile is just a macro definition; it sets the OBJS variable to two object filenames. 
This will be important later. For now, take note of how you define the macro and also how you reference it 
later ($ (OBJS)). 


The# in the next line denotes a comment. 


The next item in the Makefile contains its first target, a11. The first target is always the default, the target 
thatmake wants to build when you run make by itself on the command line. 


The rule for building a target comes after the colon. For a11, this Makefile says that you need to satisfy 
something called myprog. This is the first dependency in the file; a11 depends on myprog. Note that 
myprog can be an actual file or the target of another rule. In this case, itis both (itis the rule for a11 and 
the target of OBJS). 


To build myprog, this Makefile uses the macro $ (OBJS) in the dependencies. The macro expands to 
aux.o and main.o, SO myprog depends on these two files (they must be actual files, because there aren't 
any targets with those names anywhere in the Makefile). 


This Makefile assumes that you have two C source files named aux.c and main.c in the same directory. 
Runningmake on the Makefile yields the following output, showing the commands that make is running: 


ele: -C -O aux.o aux.c 
CC -c -ọO main.o main.c 
cc -0o myprog aux.o main.o 


So how does make know how to go from aux.c to aux.o? After all, aux.c is not in the Makefile. The 
answer is that make follows its built-in rules. It knows to look for a .c file when you wanta .o file, and 
furthermore, it knows how to run cc-c on that .c file to get to its goal of creating a . file. 


The final step of getting to myprog isa little tricky, but the idea is clear enough. Once you have the two 
object files in $ (OBJS) , you can run the C compiler according to the following line (where $ (CC) expands 
to the compiler name): 


$(CC) -o myprog $(OBJS) 


Pay special attention to the whitespace before $ (cc). This is atab. You must put a tab before any real 
command, and it must be on a line by its own. Watch out for this: 


Makefile:7: *** missing separator. Stop. 


An error like this means that the Makefile is broken. The tab is the separator, and if there is no separator or 
there's some other interference, you will see this error. 


Staying up to Date 


One last make fundamental is that targets should be up to date with their dependencies. If you type make 
twice in a row for the preceding example, the first command builds myprog, but the second command 
yields this output: 


make: Nothing to be done for ‘all'. 


For this second time through, make looked at its rules and noticed that myprog already exists. To be more 
specific,make did not build myprog again because none of the dependencies had changed since the last 
time it built myprog. You can experiment with this as follows: 


1. Runtouch aux.c. 


2. Runmake again. This time, make figures out that aux.c is newer than the aux.o already in the 
directory, so it must compile aux. o again. 


3. myprog depends on aux.o, and now aux.o is newer than the preexisting myprog, SO make must 
createmyprog again. 


This type of chain reaction is very typical. 
Command-Line Options 
You can get a great deal of mileage from make if you know how its command line options work. 


The most common make option is specifying a single target on the command line. For the preceding 
Makefile, you can run make aux.o if you want only the aux.o file. 


You can also define a macro on the command line. Let's say that you want to use a different compiler 
calledmy_bad_cc. Try this: 


make CC=my_bad_cc 


Here,make uses your definition of Cc instead of its default compiler, cc. Command-line macros come in 
handy when you're testing out preprocessor definitions and libraries, especially with the CFLAGS and 
LDFLAGS macros explained later. 


You don't even need a Makefile to run make. If built-in make rules match a target, you can just ask make to 
try to create the target. For example, if you have the source to a very simple program called blah.c, try 
make blah. The make run tries the following command: 


cc blah.o -o blah 


This use of make works only for the most elementary C programs; if your program needs a library or 
special include directory, you're probably better off writing a Makefile. 


Runningmake without a Makefile is actually most useful when you aren't dealing with a C program, but 
with something like Fortran, lex, or yacc. It can be a real pain to figure out how the compiler or utility 
works, so why not let make try to figure it out for you? Even if make fails to create the target, it will 
probably still give you a pretty good hint as to how you might use the tool. 


Two more make options stand out from the rest: 


-=n Prints the commands necessary for a build, but prevents make from actually running any 
commands 


-f file Tells make toread from file instead of Makefile or makefile 


Standard Macros and Variables 


make has many special macros and variables. It's difficult to tell the difference between a macro anda 
variable, so this book uses the term macro to mean something that usually doesn't change after make 
starts building targets. 


As you saw earlier, you can set macros at the start of your Makefile. The following list includes the most 
common macros: 


CFLAGS C compiler options. When creating object code from a .c file, make passes this as an 
argument to the compiler. 


LDF LAGS Like CFLAGS, but for the linker when creating an executable from object code. 


LDLIBS If you use LDFLAGS but do not want to combine the library name options with the search 
path, put the library name options in this file. 


cc The C compiler. The default is cc. 


CPPFLAGS C preprocessor options. When make runs the C preprocessor in some way, it passes this 
macro's expansion on as an argument. 


CXXFLAGS GNU make uses this for C++ compiler flags. Like C++ source code extensions (and nearly 
everything else associated with C++), this isn't standard and probably won't work with other make 
variants unless you define your own rule. 


Amakevariable changes as you build targets. Because you don't ever set variables by hand, the following 
list includes the $. 


$@ When inside a rule, this expands to the current target. 


$* Expands to the basename of the current target. For example, if you're building blah.o, this 
expands to blah. 


The most comprehensive list of the make variables on Linux is the make info page. 


Note Keep in mind that GNU make has many extensions, built-in rules, and features that other 
variants do not have. This is fine as long as you're running Linux, but if you step off onto a Sun 
or BSD machine and expect the same stuff to work, you might be in for a surprise. 


Conventional Targets 
Most Makefiles contain several standard targets that perform auxiliary tasks related to compiles. 


=m Theclean target is ubiquitous; a make clean usually instructs make to remove all of the object files 
and executables so that you can make a fresh start or pack up the software. Here is an example rule 
for the myprog Makefile: 


clean: 
rm. -f S(OBJUS) myprog 


a A Makefile created with the GNU autoconf system always has a distclean target to remove 
everything that wasn't part of the original distribution, including the Makefile. You will see more of this 
inSection 9.2.3. On very rare occasions, you may find that a developer opts not to remove the 
executable with this target, preferring something like realclean instead. 


=» install copies files and compiled programs to what the Makefile thinks is the proper place on the 
system. This can be dangerous, so you should always run a make -n install first to see what will 
happen without actually running any commands. 


=» Some developers provide test or check targets to make sure that everything works after you 
perform a build. 


=» depend Creates dependencies by calling the compiler with a special option (—mM) to examine the 
source code. This is an unusual-looking target because it often changes the Makefile itself. This is no 
longer common practice, but if you come across some instructions telling you to use this rule, make 
sure that you do it. 


= all is often the first target in the Makefile; you will often see references to this target instead of an 
actual executable. 


Organizing a Makefile 


Even though there are many different Makefile styles, there are still some general rules of thumb to which 
most programmers adhere. 


In the first part of the Makefile (inside the macro definitions), you should see libraries and includes grouped 
according to package: 


X_INCLUDES=-I/usr/X11R6/include 
X LIB=-L/usr/X11R6/lib -1X11 -Xt 


PNG_INCLUDES=-I/usr/local/include 


PNG_LIB=-L/usr/local/lib -lpng 


Each type of compiler and linker flag often gets a macro like the following: 


CFLAGS=S (CFLAGS) $(X_INCLUDES) S$ (PNG_INCLUDES) 
LDFLAGS=S$ (LDFLAGS) $(X_LIB) $(PNG_LIB) 


Object files are usually grouped according to executables. Let's say that you have a package that creates 
executables called boring and trite. Each has its own .c source file and requires the code in util.c. 
You might see something like this: 


UTIL OBUS=util.c 


BORING _OBJS=S (UTIL_OBJS) boring.o 
TRITE OBUS=S(UTIL-OBUS) traitecc 


PROGS=boring trite 


The rest of the Makefile might look like this: 


all: $(PROGS) 


boring: S$ (BORING_OBJS) 
$ (CC) -o $@ $(BORING_OBJS) $(LDFLAGS) 


trite: S(TRITEOBUJS) 


( 
$ (CC) -o óQ S(TRITE_OBJS) S$(LDFLAGS) 


You could combine the two executable targets into one rule, but this is usually not good practice because 
you would not easily be able to move a rule to another Makefile, delete an executable, or group 
executables differently. Furthermore, the dependencies would be incorrect — if you had just one rule for 
boring andtrite,trite would depend on boring.c, and boring would depend on trite.c, and 
make would always try to rebuild both programs whenever you changed one of the two source files. 


Note If you need to define a special rule for an object file, put the rule for the object file just above the 
rule that builds the executable. If several executables use the same object file, put the object 
rule above all of the executable rules. 


8.2Debuggers 


The standard debugger on Linux systems is gdb, with DDD as a graphical user frontend. To enable full 
debugging in your programs, you must run the compiler with the -g option to write a symbol table and 
other debugging information into the executable. To start gdb on an executable named program, run this 
command: 


gdboprogram 
You should geta (gdb) prompt. To run program with the command-line arguments options, type this at 
the (gdb) prompt: 


runoptions 


If the program works, it should start, run, and exit as normal. However, if there's a problem, gdb stops, 
prints a stack trace and the failed source code, and throws you back tothe (gdb) prompt. Because the 
source code fragment often gives you a hint about the problem, you often want to print the value of a 
particular variable that the trouble may be related to (print also works for arrays and C structures): 


printvariable 


If you want to make gdb stop the program at any point in the original source code, use the breakpoint 
feature. In the following command, file isa source code file, and 1ine_num is the line number in that 
file where gdb should stop: 


breakfile:line_num 


To tell gdb to continue executing the program, use this command: 


continue 


To clear a breakpoint, type this: 


Cclearfile:line_num 


This section provides only the briefest introduction to the debugger. gdb comes with a very extensive 
manual that you can read online, print, or buy as Debugging with GDB [Stallman/Pesch/Shebs]. 


Note If you're interested in rooting out memory problems and running profiling test, try Valgrind 
(http://valgrind.kde.org/). 


8.3Lex and Yacc 
You may encounter lex and yacc when compiling programs that read configuration files or commands. 


a Lex is a tokenizer that transforms text into numbered tags with labels. The Linux version of lex is 
namedflex. You may need a -11 or -1 f1 linker flag in conjunction with lex. 


=» Yacc is a parser that attempts to read tokens according to a grammar. The GNU parser is bison; to 
get yacc compatibility, run bison -y. You may need the -1y linker flag. 


8.4Scripting Languages 


A long time ago, the average Unix systems manager didn't have to worry much about scripting languages 
other than the Bourne shell and awk. Shell scripts (discussed in Chapter 7) continue to be an important 
part of Unix, but as stated earlier, awk has faded somewhat from the scripting arena. However, many 
powerful successors have arrived, and many systems programs have actually switched from C to scripting 
languages (such as the sensible version of the whois program). Let's look at some scripting basics. 


The very first thing that you need to know about any scripting language is that the first line of a script looks 
similar to a Bourne shell script. For example, a Perl script starts out like this: 


#!/usr/bin/perl 


In Unix, any executable text file that starts with #! is a script. The pathname following this prefix is the 
scripting language interpreter executable. When Unix tries to run an executable file that starts with #!, it 
runs the program following the #!, and then sends the script file to the new program's standard input. 
Therefore, even this is a script: 


#!/bin/tail +2 
This program prints this line... 
and this line, too. 


The first line of a shell script often contains one of the most common basic script problems — an invalid 
path to the scripting language interpreter. Let's say that you named the previous script myscript. Now, 
what if tail were actually in /usr/bin on your system (not /bin)? When you tried to run myscript, 
you would get this error: 


myscript: No such file or directory 


Most Unix error messages aren't confusing, misleading, or otherwise incomplete, but this one certainly 
qualifies on all three counts. First of all, myscriptdoes exist — you can see it right in front of your eyes 
withis. Is there a problem with your filesystem? No. The system really meant to say that it couldn't find 
/bin/tail, but it just had a small communication breakdown. 


The worst case occurs when a script is in some system binary directory. You can locate the script with a 
utility like which or find, and you think that the program is in your path, yet you get that error message. 
The only real clue you have is the error message: "No such file or directory." If the script were actually 
missing from your path, the error message would be "command not found." 


Another script "gotcha" is that you shouldn't expect any more than one argument in the script's first line to 
work. That is, the +2 in the preceding example may work, but if you add another argument, the system 
mightdecide to treat the +2and the new argument as one big argument, spaces and all. This can vary 
from system to system; don't test your patience on something as insignificant as this. 


8.4.1Perl 


The most important third-party Unix scripting language today is Perl — Larry Wall's Practical Extraction 
and Report Language. This tool has been called the "Swiss Army Chainsaw" of programming tools, and it 
is capable of an incredible variety of tasks, from networking to database access. It excels at text 
processing, conversion, and file manipulation. 


It's hard to describe what Perl code looks like; it's kind of like an imperative-style mix of C, Bourne shell, 
sed,awk, LISP, and others. It can be a little hairy (using every single special character on your keyboard), 
but it is as fast and powerful as it is ugly. If you want to be on top of your Unix system, you need to know a 
little bit of this language. Have a look at Learning Perl [Schwartz] for a tutorial-style introduction; the larger 
reference is Programming Perl [Wall]. Getting a grasp of Perl is well worth it, not just because you will be 
able to do remarkable things with text files, but also because many of the concepts (such as associative 
arrays) carry over to other scripting languages. 


Nearly all Unix and Linux systems in the world have Perl installed somewhere. Most administrators 
maintain itas /usr/bin/perl, even if they insist on keeping all other third-party software away from 


/usr. There is extensive online documentation in the manual pages; the perl(1) page has an index of the 
other pages. 


If you're having trouble with a Perl script, here are some suggestions: 


= Look at it to see if you can tell when the script was written. It might require a newer version of Perl 
than you have. 


a If it's an old script, you may have to put backslashes in front of the @ characters in strings and 
comments. 


a It might require a module that you do not have (in particular, a database module). 


When all else fails, run perl -w script to turn on warning mode. 


8.4.2Python 


Python is a newer scripting language with a fairly strong following, enough to make it the implementation 
language for packages such as GNU Mailman. It offers many of the same system features as Perl, but it 
has a powerful interactive mode and a somewhat more organized object model. 


Python's executable is python, but there's no telling where it is on your system. Therefore, when you get 
a Python program, you may have to modify the first line in the script to match your Python installation. 


Although it was written for slightly older versions, Python Essential Reference [Beazley] is a good 
reference with a small tutorial at the beginning to get you started. 


8.4.30ther Scripting Languages 


There are some other scripting languages that you might encounter: 


PHP This is a powerful hypertext processing language often found in dynamic Web scripts. Some 
people use PHP for stand-alone scripts. The PHP Web site is at http://www. php.net/. 


Tcl Tcl (tool command language) is a simple scripting language usually associated with the Tk 
graphical user interface toolkit and Expect, an automation utility. Although Tcl does not enjoy the 
wide-spread use that it once did, you should not discount its power. Many veteran developers prefer 
Tcl, especially for its embedded capabilities. Refer to htto:/Avww.tcl.tk/ for more information on Tcl. 


m4 This is a macro processing language. 
Ruby Object-oriented fanatics enjoy programming in this language (http://www. ruby-lang.org/). 


Matlab This is a commercial matrix and mathematical programming language and library. Thereisa 
very similar open source project called octave. 


Mathematica, Maple These are more commercial mathematical programming languages with 
libraries. 


8.5Java 


Although some application programmers love Java, Unix systems programmers avoid it, and you may not 
even need it on your system. However, you should still have an idea of how it works on a typical Linux 
system. 


There are two kinds of Java compilers: native compilers for producing machine code for your system (like 
a C compiler), and bytecode compilers for use by a bytecode interpreter (usually called a virtual machine, 
even though it's more of an abstract machine). You will invariably encounter bytecode on Linux. 


Bytecode files end in .class. The Java runtime environment (JRE) contains all of the programs you need 
to run Java bytecode. To run a bytecode file, use this command: 


javafile.class 


You may also encounter bytecode files that endin . jar, which are collections of archived .class files. 
To run a . jar file, use this syntax instead: 


java -jar file.jar 


Sometimes you need to set the JAVA_HOME environment variable to your Java installation prefix. If you're 
really unlucky, you may need to use CLASSPATH to include any directories containing classes that your 
program expects. This is a colon-delimited set of directories like the regular PATH variable for executables. 


If you need to compile a . java file into bytecode, you need the Java Development Kit (JDK). You can run 
thejavac compiler from JDK to create some .class files: 


javacfile.java 


JDK also comes with jar, a program that can create and pick apart . jar files. It works like tar. 


8.6Assembly Code and How a Compiler Works 


If you want to use a compiler's advanced features, you should have an idea of how the compiler operates. 
Here is a brief summary: 


1. The compiler reads a source code file and builds an internal representation of the code inside the 
file. If there's a problem with the source code, the compiler states the error and exits. 


2. The compiler analyzes the internal representation and generates assembly code for the target 
processor. 


3. Anassembler converts the assembly code into an object file. 
4. The linker gathers object files and libraries into an executable. 


You may be specifically interested in steps 2 and 3 of this process. Assembly code is one step away from 
the raw binary machine code that the processor runs; it is a textual representation of the processor 
instructions. Here is an excerpt of a program in x86 assembly code: 


Py ioe 
movl -—8(%ebp),%eax 
imull -16(%ebp),%eax 
movl -4(%ebp) , sedx 
addl teax, tedx 
movl %edx,-12 (%Sebp) 
incl -16(%ebp) 
jmp: «3 
soze. wom Apy] 


Each line of assembly code usually represents a single instruction. To manually generate assembly code 
from a C source file, use the compiler's -S option: 


CO => =O Progo progg 


Here, porog .c isthe C source file and prog .s isthe assembly code output. You can turn an assembly 
code file into an object file with the assembler, as: 


as =0 progd.Oprog.s 


For more information about x86 assembly code, see The Art of Assembly Language [Hyde]. RISC 
assembly code is a little more comprehensible; see MIPS RISC Architecture [Kane]. If you are interested in 
how to design and implement a compiler, two good books are Compilers: Principles, Techniques, and 
Tools [Aho 1986] and Modern Compiler Implementation in ML [Appel]. 


Chapter 9: Compiling Software From Source Code 


Overview 


Most nonproprietary third-party Unix software packages come as source code that the administrator can 
compile and install. One reason for this tradition is that Unix has so many different flavors and 
architectures that it would be difficult to distribute binaries for all possible platform combinations. 
Widespread source code distribution throughout the Unix community encouraged users to contribute bug 
fixes and new features to the software, and eventually this gave rise to the term "open source." 


You can get everything you see on a Linux system comes as source code — this means everything from 
the kernel and C library to the Web browsers. This means it is possible to update and augment your entire 
system by (re-)installing parts of your system from the source code. However, you probably shouldn't 
update your machine by installing everything from source code unless you really enjoy the process. Linux 
distributions provide easier means to update core parts of the system (such as the programs in /bin). 


Don't expect your distribution to provide everything for you. When you install binary packages from a 
distribution, you have no control over configuration options, including where the software goes. Some 
packages are not available as binaries, and furthermore, binary packages will not necessarily match the 
shared libraries on your system (see Section 8.1.4). 


You should understand everything in Chapter 8 before proceeding with this chapter. Installing a package 
from source code usually involves the following steps: 


1. Unpacking the source code archive. 
2. Configuring the package. 
3. Runningmake to build the programs. 


4. Runningmake install to install the package. 


9.1 Unpacking Source Packages 


A package's source code distribution usually comes aS a .tar.gz or .tar.bz2 file, and you should 
unpack the file as described in Section 1.18. 


Before you unpack, though, verify the contents of the archive with tar tvf, because some packages 
don't create their own subdirectories in the directory where you extract the archive. Output like the 
following means that the package is probably okay to unpack: 


package-1.23/Makefile.in 
package-1.23/README 
package-1.23/main.c 
package-1.23/bar.c 


However, you might see that not all of the files are in a common directory (like package-1.23 in the 
preceding example): 


Makefile 
README 
main.c 


Extracting an archive like this one can leave a big mess in your current directory. To avoid this, create a 
new directory and cd there before extracting the contents of the archive. 


Watch out for a test listing with absolute pathnames like this: 


/etc/passwd 
/etc/inetd.conf 


You likely won't come across anything like this, but if you do, remove the archive from your system, 
because it probably contains a Trojan horse or some other malicious code. 


9.1.1Where to Start 


After you extract the contents of a source archive and have a bunch of files in front of you, try to get a feel 
for the package. In particular, look for these files: README and INSTALL. 


Always look at any README files first. They often contain a description of the package, a small manual, 
installation hints, and other useful information. Many packages also come with INSTALL files that contain 
instructions on how to compile and install the package. Check for special compiler options and definitions. 


Apart from the README and INSTALL files, you will find other package files that fall into roughly three 
categories: 


a Files relating to the make system, such as Makefile,Makefile.in, and configure. Some 
packages come with a Makefile that you may need to modify. However, most modern packages use 
GNU autoconf and come with a configure script to generate a Makefile from Makefile.in based 
on your system settings and configuration options. 


=» Source code files ending in .c,.h, or .cc. C source code files may appear just about anywhere ina 
package directory. C++ source code files usually have .cc,.C, or .cxx suffixes. 


a» Object files ending in .o or binaries. Normally there aren't any object files in source code distributions. 
However, you may find object files in rare cases when the package maintainer is not allowed to 
release certain source code, and you may need to do something special to use the object files. In 
most cases, object (or binary executable) files in a source distribution mean that the package wasn't 
put together too well, and you should run make clean to make sure that you get a fresh compile. 


9.2GNU Autoconf 


Even though C source code is usually fairly portable, there are still differences on each platform that make 
it impossible to compile most packages with a single Makefile. An early solution to this problem was to 
provide individual Makefiles for every operating system, or to provide a Makefile that was easy to modify. 
This approach somehow evolved into scripts that generate Makefiles based on an analysis of the system 
used to build the package. 


GNU autoconf is now the most popular system for automatic file generation. Packages using this system 
come with files named configure,Makefile.in, and config.h.in. To generate a Makefile, run 
configure to check your system for prerequisites: 


./configure 


You should get a lot of diagnostic output like this: 


checking build system type... 1586-pc-linux-gnu 

checking host system type... 1586—-pc=Linux-gnu 

checking for a BSD-compatible install... /bin/install -c 
checking whether build environment is sane... yes 


Note configure creates a cache file (config. cache) so that it does not need to run certain tests 
more than once. In addition, configure consults config.cache on subsequent runs (this may 
come up if your system requires special options). 


If all goes well, configure creates one or more Makefiles and config.h: 


configure: creating ./config.status 
config.status: creating Makefile 


config.status: creating config.h 


Note If you're looking for a package to test, get the coreutils package from the GNU FTP site 
(ftp://ftp.gnu.org/pub/gnu/coreutils/). Coreutils includes common system programs like 1s and 


cat. 


Now you can type make to compile the package. A successful configure step doesn't necessarily mean 
that the make step will work, but the chances are pretty good (see Section 9.6 for troubleshooting failed 
configures and compiles). 


After the build completes, you might want to try running a program from the package, just to see if it works. 
To install the program, run this command: 


make install 


9.2.1configure Script Options 


Theconfigure script that comes with autoconf-enabled packages has several useful options. The most 
important of these is the installation directory. By default, the install target uses a prefix of 
/usr/local — that is, binary programs go in /usr/local/bin, libraries go in /usr/local/lib, and 
so on. 


If you want to use a prefix other than /usr/local, run configure with this option: 


./configure --prefix=new_prefix 


For example, if you specify --prefix=/tmp/test, Amake install puts binaries in /tmp/test/bin, 
libraries in /tmp/test/lib, and so on. 


Most versions of configure have a —-help option that lists other configuration options. Unfortunately, 
the list is usually so long that it's sometimes hard to figure out what might be important. Here are some 
other options: 


--bindir=directory Installs executables in directory. 
--sbindir=directory Installs system executables in directory. 
--libdir=directory Installs libraries in directory. 


—-disable-shared Prevents the package from building shared libraries. Depending on the library, 
this can save hassles later on (see Section 9.4.1). 


-—-with-package=directory Tells configure that package Isin directory. This is handy 
when a necessary library is in a nonstandard location. Unfortunately not all configure scripts 
recognize this type of option, and furthermore, it can be difficult to determine the exact syntax. 


You can create separate build directories if you want to experiment with some of these options. To do this, 
create a new directory anywhere on the system, and from that directory, run the configure script in the 
original package source code directory. You will find that configure then makes a symbolic link farm in 
your new build directory, where all of the links point back to the source tree in the original package 
directory. Some developers actually prefer that you build packages this way. 


9.2.2Environment Variables 


You can influence configure with environment variables that the configure script puts into make 
variables. The most important variables are CPPFLAGS,CFLAGS, and LDFLAGS. Be aware that 
configure Can be very picky about environment variables. For example, you should always use 
CPPFLAGS Instead of CFLAGS for header file directories, because configure often runs the preprocessor 
independently of the compiler. 


Inbash, the easiest way to send an environment variable to configure is by placing the variable 
assignment in front of ./configure onthe command line. For example, to define a DEBUG macro for the 
preprocessor, use this command: 


CPPFLAGS=-DDEBUG ./configure 


Environment variables are especially handy when configure doesn't know where to look for third-party 
include files and libraries. For example, to make the preprocessor search in include_dir, run this 
command: 


CPPFLAGS=-Iinclude_dir ./configure 


As shown in Section 8.1.5, to make the linker look in 1ib_ dir, use this command: 


LDFLAGS=-Lilib_ dir ./configure 


Ifl1ib_dir has shared libraries (see Section 8.1.4), the previous command probably won't set the runtime 
dynamic linker path. In that case, use the -rpath linker option in addition to -L : 


LDFLAGS="-Liib_dir -Wl,-rpath=lib_dir" ./configure 


Be careful when setting variables. A small slip can trip up the compiler and cause configure to fail. Let's 
say that you forget the - in -1I, asin this example: 


CPPFLAGS=Iinclude_dir ./configure 


This yields an error like this: 


checking for C compiler default output... configure: error: C compiler 
cannot create executables 
See ‘config.log' for more details. 


Digging through config.1log yields this: 


configure:2161: checking for C compiler default output 
configure:2164: gcc LIinclude_dir conftest.c >&5 
gcc: Iinclude dir: No such file or directory 


9.2.3Autoconf Targets 


In addition to the standard all and install, an autoconf-generated Makefile has these targets: 


make clean As described in Section 8.1.5,make clean removes all object files, executables, and 
libraries. 


make distclean This is similar to make clean, but it removes all automatically generated files, 
including Makefiles, config.h,config.log, and so on. The idea is that the source tree should look 
like a newly unpacked distribution after running make distclean. 


make check Some packages come with a battery of tests to verify that the compiled programs work 
properly; the command make check runs the tests. 


make install-strip Thisis like make install, but it strips the symbol table and other 
debugging information from executables and libraries when installing. Stripped binaries require much 
less space. 


9.2.4Autoconf Log Files 


If something goes wrong during the configure process and the cause isn't obvious, you can examine 
config.log to find the problem. Unfortunately, config.1log is often a gigantic file, making it difficult to 
locate the exact source of the problem. 


The best approach is to go to the very end of config.1log (for example, by pressing G in less), and then 
page back up until you see the problem. However, there is still a lot of stuff at the end, because 
configure dumps its entire environment there, including output variables, cache variables, and other 
definitions. 


You can also try running a search for the error message, starting from the end of the file. For example, 
let's say that configure fails, terminating with this message: 


See 'config.log' for more details. 


Search backward for the string "for more details" (or "config.log," or anything that looks reasonably 
unique); it's a pretty good bet that the error is nearby. 


9.30ther Systems 


If a package does not use GNU autoconf for platform-specific configuration, you should expect that the 
package uses one of these three systems: 


» A customizable Makefile and/or config.h. At one time, administrators had to configure nearly all 
third-party Unix software by editing Makefiles. GNU autoconf has made this practice nearly extinct, 
but you still may come across custom configuration in older packages or very Linux-specific 
packages. 


a Imakefiles. Many X Window System applications use imake to create Makefiles from Imakefiles. 
a "Anybody's guess." Some developers just have to be different. 


In addition, many packages, including those using the three types listed above, now rely on the pkg- 
config program for compiler and linker options. You will see how these work in Section 9.3.3. 


9.3.1Custom Makefiles 


When you need to customize a Makefile, make a copy of the original Makefile (to Makefile.dist, for 
example) before changing anything. This ensures that you can always refer to a pristine original when you 
delete or alter a line that you shouldn't have touched. 


A typical custom Makefile might look like this: 


# CC=gcc 

CELAGO==0 

# use -O to enable optimizer 
# CFLAGS=—O 


DESTDIR=/us7r/ local 
BINDIR=S (DESTDIR) /bin 
MANDIR=S (DESTDIR) /man/manl 


To enable X11 support, use -DHAS_X11 
X1LIFLAGS=—-DHAS_ X11 
XINC=-I/usr/X11R6/include 
XLIB=-L/usr/X11R6/lib -1X11 -1Xt 


Don't edit below this line 


Heo HE tE SHE HE HE 


As you learned in Chapter 8,# is a comment in a Makefile. Custom Makefiles often include macros that 
are commented out; and uncommenting those macros allows you to override defaults and add extra 
features. 


ThexINc and XLIB macros in the preceding example illustrate use of a third-party library. Macros like 
DESTDIR are analogous to the -—-prefix parameter in the GNU autoconf system. Don't expect all 
Makefiles to have such a mechanism, though. 


Finally, some particularly old configuration files ask you to specify whether your system is a BSD-like or 
SVR4 (System V Release 4) system. At one time, most Unix flavors were derived from one or the other. 
Linux leans more toward SVR4, but depending on the nature of the package, there are many BSD-isms on 
Linux that you might need to take into account. 


9.3.2ilmake 


Many X Window System applications use imake to create Makefiles from Imakefiles. The Imake system 
attempts to produce valid Makefiles from Imakefile template files with the help of predefined default 
values for the C compiler, options, and so on. The imake program uses the C preprocessor for its dirty 
work. 


Imake is standard on any platform that has an X Window System installation (usually located in 
/usr/X11R6). Imake's ubiquity is its only advantage. The system's disadvantages are that it's not very 


flexible (it usually requires installing packages with a /usr/X11R6 prefix) and it's not very smart (it doesn't 
dig around in your system and verify that things actually work as advertised). 


To build and install a package that comes with an Imakefile, do the following: 
1. Runxmkmf or xmkmf -a (use -a if there are subdirectories) to create the valid Makefile. 


2. Runmake to compile the package. 
3. Runmake install to install the executables and libraries. 


4. Runmake install.man to install the manual pages. 


X Resource Files 


Many packages that have Imakefiles also use application default ("app-defaults") files to store default 
configuration data. A program window missing an app-defaults file may look "plain" or extremely bad. 


App-defaults files are usually in /usr/X11R6/1ib/X11/app-defaults. Some packages give you an 
option for installing their app-defaults file in a place other than the default. Unfortunately, most do not, and 
you might have to resort to this ugly trick in order to install the package in a nonstandard location: 


1. Identify the programs to be installed with make -n install. 


2. Identify the app-defaults file (it usually ends with .ad). Your make -ninstall from the previous 
step should tell you the name of the installed file, though in the make -n install output the app- 
defaults filename normally does not have the . ad extension. 


Install the program by hand. 
Install the app-defaults file without a . ad extension. 


Rename the program with a .bin extension. 


oo Of œ 


Write a wrapper script like the following and name it program without the .bin extension 
(app_defaults_path Is where you installed the app-defaults file, and 
prog_path/program.bin is the binary program): 


#!/bin/sh 
XUSERFILESEARCHPATH=app_defaults_path/%N%S:SXUSERFILESEARCHPATH 
export XUSERE ILESKARCHPATH 

exec prog_path/program.bin 


9.3.3pkg-config 


The number of third-party libraries has risen dramatically in recent years. Because keeping all add-on 
libraries in a common location can be messy, you may wish to install each with a separate prefix. 
However, doing so can lead to problems when building packages that require these third-party libraries. 
For example, if you want to compile OpenSSH, you need the OpenSSL library. How do you tell the 
OpenSSH configuration process the nonstandard location of the OpenSSL header files and libraries? 


Many libraries now use the pkg-config program not only to advertise the locations of their include files 
and libraries, but also to specify the exact flags that you need to compile and link a program. The syntax is 
as follows: 


pkg-config options packagel package2 


For example, to find the include files for OpenSSL, you can run this command: 


pkg-contig --cflags openssl 


The output should be something like this: 


—-T/opt/openssl/include 


If you want the linker flags, use --1dflags instead. This should yield a library location and the names of 
the individual libraries, as in this example for OpenSSL: 


=L / Opt /openss 1/7 ib =Lss1 =—Lerypto:—ld) 


To see all libraries that pkg-config knows about, run this command: 


PKO=-cConiag “—-—-Listaali 


If you look behind the scenes, you will find that pkg-config finds package information by reading 
configuration files that end with .pc. For example, here is openssl.pc, from the OpenSSL socket library: 


prefix=/opt/openssl 
exec_prefix=${prefix} 
libdir=$ {exec_prefix}/lib 
includedir=S {prefix}/include 


Name: OpenSSL 

Description: Secure Sockets Layer and cryptography libraries and tools 
Version: 0.9.7b 

Requires: 

Lips -=Le (ibd) ssl —Loerypce. =lal 

Ciilags> =<Is{i1neludedir} 


You can change this file if you like — for example, you may wish to add -W1,-rpath=${libdir} tothe 
library flags to set a runtime dynamic linker path. However, the bigger question is how pkg-config finds 
the. pc files in the first place. What happens is that pkg-config looks in the 1ib/pkgconfig directory 
of its installation prefix. A pkg-config installed with a /usr/local prefix looks in 
/usr/local/lib/pkgconfig. 


Unfortunately, okg-config does not read any .pc files outside of its installation prefix by default. The 
previous OpenSSL example . pc file would reside at /opt /openss1/lib/pkgconfig/openssl.pc, 
almost certainly out of the reach of any stock pkg-config Installation. 


There are two basic ways to show pkg-config the .pc files that are located outside of the pkg-config 
installation prefix: 


a» Make symbolic links (or copies) from the actual .pc files to the central pkgconfig directory. 


a Set your PKG_CONFIG_PATH environment variable to include any extra pkgconfig directories. This 
strategy does not work well on a system-wide basis. 


9.4Iinstallation Practice 


Knowinghow to build and install software is good, but knowing when and where to install your own 
packages is what proves to be useful. Linux distributions try to cram in as much software as possible on 
installation. You should always ask yourself if you should install a package by yourself instead. 


Here are the advantages of doing it yourself: 
=» You can customize package defaults. 
=» Custom software usually survives operating system upgrades. 
a When installing a package, you often get a much clearer picture of how to use the package. 
= You always get the latest releases. 
a It's easier to back up a custom package. 
a It's easier to distribute self-installed packages across a network. 
These are the disadvantages: 
a It takes time. 
a If you dont actually use the package, you're wasting your time. 
=» There is a potential for misconfiguring packages. 


Consider the C library and the coreutils package (1s,cat, and so on) — there really is no point in building 
these by yourself unless you have some real obsession with the build process. On the other hand, if you 
have a vital interest in network servers such as Apache, the best way to get complete control is to install 
the servers yourself. 


9.4.1Where to Install 


The default prefix in GNU autoconf and many other packages is /usr/local, the traditional directory for 
locally installed software. Operating system upgrades ignore /usr/local, so you won't lose anything 
installed there during an operating system upgrade. 


For small local software installations, /usr/local Is fine. However, if you have a lot of custom software 
installed, this can turn into a terrible mess. Thousands of odd little files can make their way into the 
/usr/local hierarchy, and you may have no idea where the files came from. 


There are several schemes that help manage software packages. | like the Encap system 
(hitp:/Awww.encap.org/) because it requires no extra software. 


Encap 
Encap abuses symbolic links, but that's a small price to pay for the features you get: 
=» You can identify everything in /usr/local by package name. 


= You can keep old versions of a software package intact, so you can always downgrade, run the old 
version, and or use the old version as a model. 


a |t is easy to remove old versions and entire packages. 

=» You can easily transport packages to another partition or another machine. 

=» Encap requires very little extra configuration. 
The main idea behind Encap is that you encapsulate a package installation before making it available in 
/usr/local. To start, you must have a common installation root (most administrators use 


/usr/local/encap or /opt; this book uses /opt because it's shorter). 


Let's say that you are installing bogon-3.2.3. Here's what you would do with Encap: 
1. Configure, build, and install the package with an installation prefix that reflects the package and 


version. For example, the package uses GNU autoconf, so run this command: 


./configure --prefix=/opt/bogon/bogon-3.2.3 


2. Goto /opt/bogon and make a symbolic link to the install you just made from default: 


cd /opt/bogon 
Im -8 /opt/boqon/bogon-3.2.3 default 


3. If there are binary programs in /opt /bogon/default/bin, run this command: 


ln -s /opt/bogon/default/bin/* /usr/local/bin 


4. If there are binary programs in /opt /bogon/default/sbin, run this command: 


ln =s /opt/bogon/default/sbin/* /usr/local/sbin 


5. Run the appropriate variant of the following command for each section in which there are manual 
pages (the example works for man section 1): 


In -s /opt/bogon/default/man/manl/* /usr/local/man/manl 


Now the bogon package binaries and manual pages are available in /usr/local via symbolic links. 
Furthermore, you can now do the following: 


= If you ever need to know the originating package of an item in /usr/local/bin, you need only run 
ls -1 onthe link to see where it points. 


=» To remove a package, you can remove the symbolic links and its directory in /opt. For example, to 
remove the bogon symbolic links in /usr/local/bin, run this: 


cd /opt/bogon/default/bin 
fOr Tile an “7 -do 

rm -f /usr/local/bin/S$file 
done 


Should You Compile Shared Libraries? 


Shared libraries (see Section 8.1.4) cause trouble with custom installations, primarily because a shared 
library upgrade can break a large number of programs at once. For this reason, you should never make 
links to shared libraries in /usr/local/1lib when using Encap. Leave shared libraries in their installed 
location. 


Even with this precaution, you still have these problems: 


a It's hard to tell if you can actually delete an old shared library, because some program may depend on 
the library. 


a It's difficult to link programs against the shared library. You must use the -W1, -rpath=path linker 
option. 


a» When linking against a shared library, you should specify a path that includes a version number. For 
example, use /opt /bogon/bogon-3.2.3/1lib instead of /opt /bogon/default. You risk library 
version mismatches otherwise. 


In fact, shared libraries can cause so many problems that you should always think twice before installing 
them. If the library is fairly small (such as libpng and 1ibtiff), there's no problem in opting for static 
libraries over shared libraries. Use this configure option to disable shared libraries for packages that use 
GNU autoconf: 


./configure -—-disable-shared 


Encap, pkg-config, and Shared Libraries 


You can't get away with static libraries for everything. Enormous libraries such as GNOME components 
demand shared libraries. In the worst cases, you need to slug it out with LDFLAGS Makefile and 
environment variables. Happily, many larger libraries also use pkg-config, described in Section 9.3.3. 


You can make Encap work with pkg-config if you put in a little extra effort. To set it up, you need to 
Know which 1ib/pkgconfig directory pkg-config searches for .pc files. Let's say that you're trying to 
install the shared libraries from bogon-3.2.3 with Encap: 


1. 
2: 


Configure, compile, and install the package as usual. 


Carry out the Encap steps discussed earlier. In particular, you need the default symbolic link to 
point to bogon-3.2.3. 


Go to /opt /bogon/default/lib/pkgconfig. You should see a file named bogon. pc. 


Editbogon.pc. Look for this line: 


Libs: -ls{iibdir} —Lboogon 
Change that line to reflect a runtime link path: 
Libs: —-W1l,-rpath=S{libdir} -LS{libdir} —lbogon 


Now tell pkg-config about the package. If pcpath is where pkg-config keeps its . pc files (for 
example,ocpath could be /usr/local/lib/pkgconfig), create a symbolic link: 


In =s /opt/bogon/default/lib/pkgconfig pcpath 


Testokg-config ON bogon: 


pkg-config —-libs: bogon 


The output should include -W1,-rpath. 


9.5Applying a Patch 


Every now and then, you might get a patch that you need to apply against source code to fix bugs or add 
features. You may also see the term diffused as a synonym for patch, because the diff program 
produces the patch. 


The beginning of a patch looks something like this: 


=== SUC/ Llesg 2003=07=17 Dae 29s12.000000000 +0100 
Pr BrC/rile.c Z2U03=09-Le LOt22217. 000000000 +0100 
aU —2,16. 42,12. Uu 


Patches usually contain alterations to more than one file. Search the patch for three dashes in a row (---) 
if you want to see the files that have alterations. 


You should always look at the beginning of a patch to determine the required working directory. Notice that 
the preceding example refers to src/file.c. Therefore, you should change to the directory that contains 
src before applying the patch, notto the src directory itself. 


To apply the patch, run the patch command: 


patch -p0 < patch file 


If everything goes well, patch exits without a fuss, leaving you with an updated set of files. However, 
watch out if patch asks you this question: 


File to patch: 


This usually means that you are not in the correct directory. However, it could also indicate that your 
source code does not match the source code in the patch. In this case, you're probably out of luck — even 
if you could identify some of the files to patch, others would not be properly updated, leaving you with 
source code that you cannot compile. 


In rare cases, you might come across a patch that refers to a package version like this: 


=== packadge-3.42/srce/file.c.orig 2003-07-17 ba 2907 l2.000000000 +0100 
+++ package-3.42/src/file.c 2003-09-16 102227 17.000000000' +UL00 


If you have a slightly different version number (or you just renamed the directory), you can tell patch to 
strip leading path components. Let's say that you were in the directory that contains src (as before). To 
makepatch ignore the package-3.42/ part of the path (that is, to strip one leading path component), 
use-p1: 


batch -p1 < patch file 


9.6 Troubleshooting Compiles and Installations 


lf you understand the difference between compiler errors, compiler warnings, linker errors, and shared 
library problems as described in Chapier 8, then you should not have too much trouble fixing many of the 
glitches that arise when building software. This section covers some other common problems. 


Before getting into specifics, you should learn to read make output. You first need to know the difference 
between an error and an ignored error. The following is a real error that you need to investigate: 


make: *** [target] Error 1 


However, some Makefiles suspect that an error condition may occur but know that these errors are 
harmless. You can usually disregard any messages like this: 


make: *** [target] Error 1 (ignored) 


Furthermore, GNU make often calls itself many times in large packages, with each instance of make in the 
error message marked with [N], where Nis a number. You can often find the error quickly by looking at 
themake error that comes directly after the compiler error message. Here is an example: 


[compiler error message involving file.c] 
make[3]: *** [file.o] Error 1 


make[3]: Leaving directory '/home/src/package-5.0/src' 
make [2] mee [all] ECror z 

make[2]: Leaving directory '/home/src/package-5.0/src' 
make[l]: *** [all-recursive] Error 1 

make[1]: Leaving directory '/home/src/package-5.0/' 
Makes *** fall] Error z 


The first three lines practically give it away — the trouble centers around file .c located in 
/home/src/package-5.0/src. Unfortunately, there is so much extra output that it can be difficult to 
spot the important details. Learning how to filter out the subsequent make errors goes along way toward 
digging out the real cause. 


9.6.1Specific Errors 
Problem: Compiler error message: 


Src.,ctz2: conflicting types for Titem” 
/usr/include/file.h:47: previous declaration of '‘item' 


Explanation and Fix: The programmer made an erroneous redeclaration of item online 22 of src.c. You 
can usually fix this by removing the offending line (with acomment, an #ifdef, or whatever works). 
Problem: Compiler error message: 


srco.c:!:37: time t' undeclared (iirst use this function) 


SrO: o ls parse crror bDerore ~~..." 


Explanation and Fix: The programmer forgot a critical header file. The manual pages are the best way to 
find the missing header file. First, look at the offending line (in this case, it's line 37 in src.c). It's probably 
a variable declaration like the following: 


Lime © viz 


Search forward for v1 in the program for its use around a function call. For example: 


vl = time (NULL); 


Now run man 2 time Orman 3 time to look for system and library calls named time (). In this case, 
the section 2 manual page has what you need: 


SYNOPSIS 
tincluüude <time. h> 


time t time(time -t *E); 


This means that time () requires time.h. Place #include <time.h> atthe beginning of src.c and 
try again. 


Problem: Compiler (preprocessor) error message: 


src.c:4:pkg.h: No such file or directory 
(long list of errors follows) 


Explanation and Fix: The compiler ran the C preprocessor on src.c, but could not find the pkg .h include 
file. The source code likely depends on a library that you need to install, or you may just need to provide 
the compiler with the nonstandard include path (see Section 9.2.2 if the package uses GNU autoconf). In 
most cases, you just need to add a -I include path option to the C preprocessor flags (CPPFLAGS). Keep 
in mind that you may also need a -L1 linker flag to go along with the include files. 


If you are using a graphics application and see Xm and Mrm in the error message, you are missing the Motif 
library. LessTif is a free replacement for Motif; if it is not on your system, you can get it at 
http://www. lesstif .org/. 


If it doesn't look as though you're missing a library, there's also an outside chance that you are attempting 
a compile for an operating system that this source code does not support. Check the Makefile and README 
files for details about platforms. 


Problem:make error message: 


make:prog: Command not found 


Explanation and Fix: To build the package, you need prog on your system. If prog is something like cc, 
gcc, or 1d, you don't have the development utilities installed on your system. On the other hand, if you 
thinkprog is already installed on your system, try altering the Makefile to specify the full pathname of 


Prog: 


In rare cases, make builds prog and then uses prog immediately, assuming that the current directory (.) 
is in your command path. If your $PATH does not include the current directory, you can edit the Makefile 
and change prog to ./prog. Alternatively, you could append .to your path temporarily. 


Problem:Undefined symbols that start with SM_ and ICE_. 


Explanation and Fix: Some packages that link against X Window System libraries also need the -1 sM and 
—11ICE linker flags. 


Chapter 10: Maintaining the Kernel 


Overview 


As mentioned in Section 2.2, the kernel is the core of the operating system. In some respects, the Linux 
kernel is no different than any other software package. You can configure, build, and install the kernel 
because it comes as source code. However, the procedure for doing so is substantially different than for 
any other software package because the kernel runs like no other package. 


There are four basic topics pertaining to kernel management: 


Configuring and compiling a new kernel Your end goal here is to build an image file, ready for the 
boot loader. 


Manipulating loadable kernel modules You don't have to build the kernel as one massive chunk of 
code. Kernel modules let you load drivers and kernel services as you need them. 


Configuring a boot loader such as GRUB or LILO Your kernel is useless if you can't load it into 
memory. 


Learning miscellaneous utilities and procedures There are many facilities that tweak runtime 
kernel parameters and extend kernel features. You have already seen some examples of these in 
previous chapters, including the /proc filesystem and the iptables command. 


10.1Do You Need to Build Your Own Kernel? 


Before running head-first into a kernel build, you need to ask yourself if it is worth it. Administrators who 
compile their own kernels have the following goals in mind: 


a Installing the latest drivers 
a Using the latest kernel features (especially with respect to networking and filesystems) 
a Having fun 


However, if you need a driver or feature, and your distribution offers a straightforward upgrade, you might 
opt for that instead, for several reasons: 


=» You can make mistakes in configuring your own kernel, and your distribution probably offers a well- 
rounded kernel. 


a» Compiling a new kernel takes a long time. 
a It's far too easy to mess up the boot loader, making your system unable to boot. 


Even if you choose not to build your own kernel, though, you should still learn how boot loaders and 
modules work. 


10.2What You Need to Build a Kernel 


As with any package, the kernel source includes documentation explaining what you need to compile a 
functional kernel. These three aspects of your system are particularly important for building a kernel: 


» AC compiler (gcc). Most distributions put the C compiler in a development tools package. Make sure 
that your C compiler agrees with the recommendations in the kernel source code, specifically the 
README file. Kernel developers have not been eager to adopt the latest versions of gcc. Some 
distributions have a separate compiler named kgcc for compiling the kernel. 


=» Disk space. A new kernel source tree can easily unpack to more than 100MB even before building any 
object files. 


= Arelatively fast computer with plenty of memory. Otherwise, the compile will take some time. 


Your first step is to get the source code. 


10.3Getting the Source Code 


Linux kernel versions have three components. Suppose you have kernel release 2.6.3. Here's what the 
numbers mean: 


a 2 is the major release number. 
=» 6 is the minor release number. 


= 3 is the patchlevel. Kernels with the same major and minor numbers but with different patchlevels 
belong to a release series. 


There are two different kinds of kernel releases: 


=» Production releases have minor version numbers that are even (for example, 1.2.x, 2.0.x , 2.2.x, 2.4.x, 
2.6.x, and so on). These versions are meant to be as stable as possible. Within a single production 
release series, there are no radical feature or interface changes. 


=u Development releases have minor version numbers that are odd (such as 2.5.). Don't use a developer 
release if you aren't willing to take risks. Kernel developers intentionally break code, introduce new 
features, and may inadvertently make the kernel unstable in a development series. Stay away from 
development releases unless you know exactly what you're doing. 


You can get the latest Linux kernel source code at http://www.kernel.org/. The releases have names like 
linux-version.tar.bz2 for bzip2 compression or linux-version.tar.gz for GNU Zip 


compression. Bzip2 archives are smaller. 


In addition to the full releases available in the kernel archives, you will also find patches containing just the 
changes between two consecutive kernel patchlevel releases. Unless you are obsessed with upgrading 
your kernel with every single release, you probably won't be able to do much with a patch. 


10.3.1Unpacking the Source Archive 


You can unpack kernels with zcat/bzip2 and tar. To get started, go to /usr/src and make your 
permissions less restrictive if necessary: 


cda /uüusr/src 
umask 022 


Then run one of these commands to extract the source files, depending on whether you are using bzip2 or 
GNU Zip: 


bzip2ż =—de linux- vērsionstar;bz2 | tar xví = 
Zcat Linux—version.-tar.gz | tar xvi = 


Warning If your kernel release is 2.4.18 or lower, the kernel unpacks into a directory named 1 inux 
rather than 1inux-version. Rename any existing 1inux directory before unpacking one 
of these kernels (the new name does not matter, as long as it is different than the old one). 


10.3.2A Look Around the Kernel Source 


If you've never seen the Linux kernel source code before, it helps to take a look around first. The top-level 
directory includes several subdirectories and files, including these: 


README This file is a general introduction to the Linux kernel source and what you need to get started. 
This document explains the version of the C compiler that you need to compile the kernel. 


Documentation [his directory contains a wealth of documents. Most of the documents here are in 


plain-text format, and they may describe anything from user-level programs to low-level programming 
interfaces. One of the most important files in this directory is Changes, which describes recent 


changes to the kernel and identifies utility programs you may need to upgrade to get full functionality. 


include You'll find the kernel header files in this directory. If you feel comfortable with your kernel, 
you can use the header files in include/linux as your system /usr/include/1linux header file 
set. 


arch This directory contains architecture-specific kernel build files and targets. After a kernel build on 
a PC system, the final kernel image is in arch/i386/boot/bzImage. 


10.3.3Distribution Kernels 


Linux distributions come with generic kernels intended to run on almost any processor type. These kernels 
tend to be a little larger than custom kernels, and they rely heavily on loadable modules. 


However, many of the Linux kernels that come with the various Linux distributions omit complete source 
code in their default installations, and furthermore, distribution kernels often differ from the official 
standard kernels at kernel.org. For example, Red Hat adds another component to the patchlevel to 
distinguish their kernels (for example, 2.4.20-20). If you wish to install a distribution kernel from source 
code, you need to use the distribution's kernel source package. In Red Hat Linux, the package name is 
kernel-source-version.rpm. 


If your distribution has a /usr/src/linux directory, this does not mean that you have the entire kernel 
source. It is possible that you only have the header files. The kernel source takes up dozens of megabytes; 
rundu -s/usr/src/linux for a quick check on what you have. 


10.4Configuring and Compiling the Kernel 


You need to configure the options that you want to have in your kernel before you build it. Your goal is to 
have an appropriate . config file in your kernel source distribution. Here's an excerpt of a typical 
.config file: 


# CONFIG HIGHMFM64G is not set 

# CONFIG MATH EMULATION is not set 
CONFIG _MTRR=y 

CONFIG HAVE DEC _LOCK=y 


This file isn't easy to write with a text editor, so there are several configuration utilities that generate the 
file for you. An easy, standard way to set up a kernel is to run this command: 


make menuconfig 


After some initial setup, make menuconfig runs a text-based menu interface that is shown in Figure 10- 
1. The box in the center of the screen contains the options at the current configuration level. You can 
navigate the menu with the up- and down-arrow keys. An arrow next to an item (--->) indicates a 


submenu. 
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Figure 10-1: JETE e A configuration menu. 
The bottom of the menu contains three actions that you can navigate to with the left- and right-arrow keys. 


Pressing ENTER while one of the menu items is highlighted performs whichever of the three following 
actions is also highlighted: 


Select If the current menu item leads to a submenu, the configuration system activates the submenu. 


Exit Exits the current submenu. If you are at the top-level menu, the configuration system asks 
whether you would like to save the current configuration. You can activate the Exit option by pressing 
ESC. 


Help Shows any online help for the current item. You can also get at the help by pressing the ? key. 


To get a general feel for how configuration options work, go into the Processor type and features 
submenu. You will see options such as these: 


[ ] Math emulation 
[*] MTRR (Memory Type Range Register) support 
< > /dev/cpu/microcode - Intel IA32 CPU microcode support 


You can alter an item's configuration value by moving to the item and pressing the SPACEBAR. The 
square brackets ([]) provide a simple on/off toggle: 


[*] indicates that the feature Is on. 


[ ] indicates that the feature is off. 


You may not configure on/off features as kernel modules. Active features go directly into the main kernel 
image. 


By contrast, an item with angle brackets (<>) denotes a feature that you may compile as a module. You 
can use the SPACEBAR to toggle between these values: 
<*> indicates that the feature is on. 


<M> indicates that the feature is configured as a module. 
< > indicates that the feature is off. 


An item that includes regular parentheses like these, (), is a multiple-choice option or a number that you 
can customize. Press ENTER to see the option list, or enter a number. 


When Should You Compile a Driver as a Module? 


There aren't many absolute rules for deciding which features to compile as modules and which 
features to compile as part of the main kernel image, but here are a few guidelines and rules that can 
help you: 


= Always compile your root filesystem type directly into the kernel, not as a module. Your system 
won't boot otherwise. 


=» Most administrators prefer to compile crucial disk support (such as SCSI host adapter drivers) 
directly into the kernel. There is a way to get around this with an initial RAM disk, but that is more 
an ugly hack than anything else. 


=» Most administrators also compile network interface card drivers directly into the kernel. This isn't 
as crucial as disk support, though. 


=» Compile obscure filesystems as modules. You may need them at some point, but there is no point 
in wasting memory in the meantime. 


When you are through with configuration and decide to exit, the menu system will ask if you would like to 
save the current configuration. If you choose Yes, the system backs up your previous . config file as 
.config.old and writes anew .config file. 


10.4.1Configuration Options 


Most of the configuration options are fairly self-explanatory. For example, in the menu shown in Figure 10- 
1 on page 205, the items under Device Drivers> SCSI device support > SCSI low-level drivers 
correspond to device drivers for SCSI host controllers. 


Some configuration options depend on other options, and until you activate the dependency options you 
cannot reach configuration options that have dependencies. For example, to reach the SCSI low-level 
drivers submenu, you must first activate SCSI device support in the previous menu. These 
dependencies are not always obvious; you may need to do alittle bit of experimentation to find what you're 
looking for. If you're really stumped on a dependency, go to the source. Look at arch/i386/Kconfig, 
the master file for all options and online help for the kernel configuration. 


The following sections outline the most significant kernel options that reside within the important top-level 


menus. Keep in mind that these options change with time; you may see items not listed here, or the items 
may be in a different place in the kernel that you decide to build. 


Code Maturity Level Options 

Inside this menu item you will find an option named Prompt fordevelopment and/or incomplete 
code/drivers. To see the newest (but perhaps unstable) drivers and features when perusing the rest of the 
kernel configuration menus, select this option. 


General Setup 


This section has three settings that you should turn on: 


Support for paging of anonymous memory Enables swap disk and file support 


System V IPC Enables interprocess communication 


Sysctl support Enables kernel parameter changes through /proc 


If you do not build your kernel with the support listed here, many major packages and utilities will not work, 
including the X Window System. 


In addition to these options, the General Setup menu in kernel 2.6.0 and newer versions have an option 
namedKernel .config support to make the build process save the current . config file in the kernel 
image. If you enable this option, you will also have the opportunity to enable a second option that allows 
you to turn on access to the embedded . config file via /proc/config.gz. These options increase your 
kernel size slightly, but they can come in extremely handy later on when you need to upgrade your kernel 
but can't remember what configuration options you selected. 


Loadable Module Support 


You have some control over the kernel's module loader. In general, you should always activate the kernel 
module system with the Enable loadablemodule support option, as well as activating the kernel module 
loader option, Automatic kernel module loading, sometimes called the autoloader. 


The only item in the kernel module support menu that you should be wary of is versioning support (at the 
moment, this is experimental). Kernels compiled with this option enabled may try to load modules built for 
a different kernel version, and this can cause trouble if you don't know exactly what you're doing. 


Processor Type and Features 


The Linux kernel includes a number of optimizations and other enhancements that are specific to certain 
processors. You can choose your processor with the Processor family option. Be careful — a kernel built 
for a "primitive" CPU supports advanced processors, but a kernel tailored to an advanced CPU will likely 
not work on an older (or different brand of) CPU. 


Other significant options in the processor category include: 


High memory support This is for machines with more than 2GB of physical memory. 
Symmetric multi-processing support This is for machines with more than one processor. 


MTRR support This permits optimizations that may improve certain kinds of graphics performance. 


Power Management Options 


There are two types of power management in PC hardware: the older APM (Advanced Power 
Management) and the newer ACPI (Advanced Configuration and Power Interface). You can configure a 
kernel with both varieties. Appropriate power management support is essential on notebooks to preserve 
battery life, and it is a good idea for desktops so that your machine doesn't generate too much heat and 
use too much electricity. 


Power management features enable interesting tricks with fans, processor speeds, and more. There's a lot 
to explore, but you may need additional software such as apmd or acpid to take advantage of the kernel 
features. 


Bus Options 


You shouldn't need to change the bus options for most systems. Unless you have an extremely old or odd 
system, include PCI support (and if you'd like to list and diagnose devices on the PCI bus, install lspci, 
which is a part of the pci-utils package). 


You should enable the Support for hot-pluggable devices option so that your system can take 
appropriate action when you attach and detach removable devices. 


Newer kernels include base PCMCIA (PC Card/CardBus) drivers as configuration options in this menu — 
in previous systems, PCMCIA support was completely separate from the main kernel distribution. You still 
need the PCMCIA utilities if you intend to use PC cards, and at the moment, you can still leave out 
PCMCIA kernel support here and get it from these utilities, but this may change in the future. 


Executable File Formats 


To start a process from an executable file on the disk, the kernel needs to know how to load the 
executable file, how to initialize the process in memory, and where to start running the process. An 
executable's file format determines these characteristics. Most binaries on a modern Linux system are ELF 
(Executable and Linkable Format) files. You probably do not need to support the ancient "a.out" 
executable format (you may not even have the shared libraries to support these binaries), but it does not 
take too much memory to include support, and you can build it as a module. 


Device Drivers 

Configuring device drivers is a long process due to the wide variety of devices that Linux supports. 
Unfortunately, it's during this stage that you can get bogged down with all of the options and end up 
building a kernel that is far too large because you included drivers for devices that you will never have. If 
you're in doubt about whether you are going to use any particular driver (other than a disk driver), build it 
as a module so that it does not unnecessarily take up kernel memory. 


The following sections describe the driver configuration options inside the Device drivers menu. 
Plug and Play Support 

You must enable Plug and Play support if you want any reasonably modern built-in hardware in your 
computer to work. You may also need plug-and-play support in order to use network cards, internal 


modems, and sound cards. 


Block Devices 


TheBlock devices section of the kernel configuration contains miscellaneous random-access storage 
devices, such as floppy disks and RAM disks. These devices normally hold filesystems that you directly 
attach to your current system with the mount command. The interesting drivers here include the following: 


Normal floppy disk support A driver for the PC floppy disk drive. 


Loopback device support A simple driver that maps a file into a block device. This driver is very 
useful because it allows you to mount a filesystem on a disk image. 


Network block device support A driver that allows you to use a network server as a block device. 
RAM disk support A driver that designates a chunk of physical memory as disk space. 


Initial RAM disk (initrd) support A driver that provides a special kind of RAM disk that a boot loader 
gives to the kernel as the initial / partition during the very first stages of init. Many distributions use 
an initial RAM disk for SCSI drivers and other modules that may not be in their stock kernels. 


Parallel port IDE device support A driver for certain older portable disk devices. 


ATA (IDE) Support 


You should always compile ATA (IDE) support directly into your kernel unless you know exactly what 
you're doing. There are several kinds of IDE drivers that you can enable here: 


Disk support A driver required if you want hard drives to work. Needless to say, you should compile 
this driver directly into the kernel. 


CD-ROM support A driver for ATAPI CD-ROM and DVD-ROM drives. 
Floppy support A driver for various removable-media drives with an ATAPI interface. 


SCSI emulation A driver for ATAPI CD-R and CD-RW drives. Older Linux CD-burning software 
worked exclusively with SCSI drivers; if your CD-burning software is up to date, you do not need this 


driver. 


Various chipsets Drivers for various specific chipsets. If your motherboard's IDE chipset is listed, you 
might be able to squeak out a little more performance with one of these drivers. 


SCSI Support 


Even if you have no SCSI devices, you may still need SCSI support because many Linux device drivers 
work through emulated SCSI devices. For example, if you want to use USB mass storage, you need SCSI 
disk support. 


These are the media drivers: 


SCSI disk support Covers all fixed and removable-media disk-like storage devices, except CD-ROM 
drives. 


SCSI tape support 
SCSI CD-ROM support 


SCSI generic support Allows applications talk to a SCSI device at a low level. You need SCSI 
generic support for a wide range of devices, including CD burners, changer devices on tape drives, 
SCSI scanners, and more. 


A look inside the SCSI low-level drivers submenu reveals drivers for many kinds of SCSI host controllers. 
Some of the drivers are named after the SCSI chipset on the host controller, so you may have to look 
carefully at your hardware to find a match. The low-level driver list also includes support for some parallel 
port devices, such as Zip drives. 


Note If your kernel and root partition are on a SCSI disk, compile SCSI support (including the disk and 
host controller drivers) directly into the kernel rather than as modules. This makes booting much 
easier, because you do not need to worry about loading SCSI modules from a disk that the 
kernel does not yet know how to access. 


Networking Support 


You need networking support for almost any system, even if you do not have an external network 
connection. Many applications use local network interface features. 


Your first order of business is to pick the networking devices that you need. For most Ethernet devices, 
look under Ethernet (10 or 100Mbit). There are many devices to choose from. If you're not too sure what 
you need, don't be afraid to configure several drivers as modules. In addition, if you plan to dial up to the 
Internet or use certain DSL connections, you need PPP support, along with asynchronous (serial port) or 
PPP over Ethernet (PPPoE) support. If you're not sure about PPP options, configure them as modules. 


The configuration options inside the Networking options submenu represent the trickiest part of the 
kernel network configuration. You need to select all of the options and protocols that you intend to use. 
The essential options for many packages include the following: 


TCP/IP networking 
Unix domain sockets 
Packet socket (for features such as promiscuous mode) 


Not so obvious is Network packet filtering, which is required for any kind of firewall or NAT (Network 
Address Translation, or "IP masquerading") support. Just selecting this option is not enough because, in 
general, your kernel needs all of the connection tracking, NAT, and masquerade options if you want to use 
NAT. Therefore, you must enter the Netfilter Configuration submenu and choose more options there, 
such as these: 


Connection tracking for NAT (be sure to include any protocols you need to track). 


IP tables support for firewalling and NAT. Once you enable IP tables, you get several more options, 
including various match options for filtering. You don't have to worry about most of these, except for 
Connectionstate and Connection tracking, both of which you need for NAT. That's not the end of 
the things you need for NAT — look out for and enable Full NAT and MASQUERADE target 
support. 


Packet filtering and REJECT target support for firewalls. 


Input Device Support 


You need several configuration options to support your keyboard and mouse. There are two levels of 
support. The basic drivers support the PS/2 keyboard. Also, look for PS/2 Mouse. 


Character Devices and Parallel Port Support 


Among the most important character devices are Virtual terminals (/dev/tty0,/dev/ttyl1, and so 
on). Unless you have a special type of server, you also need to put the console on the virtual terminal. 


Here are some other things to check up on: 


Serial drivers Standard PC serial ports use or emulate 8250/16550 UART chips. You almost certainly 
need this support if you plan to use a modem. 


Unix98 PTY support Some programs now expect to see the dynamic Unix98 /dev/pts/* pseudo- 
terminal devices rather than old static names like /dev/ttyp*. 


Enhanced real time clock support This makes /dev/rtc available to programs like hwclock. You 
should enable this option for any modern machine. 


/dev/agpart This is for direct-rendered graphics (GLX and DRI) on AGP graphics cards. 


Parallel printer support This gives you /dev/1p* devices. To get this option, you must go back to 
the main kernel menu and enable Parallelport support. You also need PC-style hardware for any 
standard PC. 


Note There is a section for Mice in the character device configuration, but you can probably safely 
ignore it, because the drivers there are for very old bus mice. 


Sound 


There are two parts to sound configuration: API support and drivers. The current sound system is called 
Advanced Linux Sound Architecture (ALSA), but you should also include the emulation support for the old 
OSS (Open Sound System), because many programs still use the OSS interface. 


The driver configuration for sound devices is very similar to that for network devices — many of the sound 
drivers carry the names of the sound chipset. 


USB Support 


When configuring USB, choose an interface driver first: 


UHCI and OHCI are USB 1.1 interfaces. Even if you have a USB 2.0 motherboard, you need one of 
these. If you don't know which one your motherboard supports, pick both (modules are okay); the 
Kernel will sort it out. 


EHCI is a USB 2.0 interface. 


You also want to enable the USB device filesystem option that can maintain USB device information in 
/proc/bus/usb. 


Configuring the kernel to support USB devices is fairly straightforward, but there are two gotchas: 
=» TheMass storage support option requires that you also enable SCSI and SCSI disk support. 


= Thelnterface device support (HID) option requires that you enable the input drivers described 
earlier. To get USB mouse and keyboard support, you need to enable HID input layer support in the 
USB support menu. Don't use the boot protocol drivers unless you know exactly what you're doing. 


Filesystems 


You can add support for many different filesystems to your Linux kernel configuration. However, make 


sure that you compile your primary filesystem directly into the kernel. For example, if your root partition 
uses ext2, don't modularize ext2 support. Otherwise, your system will not be able to mount the root 
filesystem and therefore will not boot. 


These are the most important filesystems: 


Second extended (ext2) The Linux standard for many years. 
Third extended (ext3) A journaled version of ext2; now the standard for many distributions. 
Reiserfs A high-performance journaled filesystem. 


DOS/FAT/NT MS-DOS- and Windows-compatible filesystems. To get MS-DOS filesystem support, 
you need to enable FAT. VFAT is the extended filesystem introduced with Windows 95. You need 
VFAT if you plan to read images from the flash memory cards in digital cameras and similar devices. 


ISO9660 A standard CD-ROM filesystem. The Linux driver includes the Rock Ridge extensions. You 
can also add the Microsoft Joliet extensions. 


UDF A newer CD-ROM and DVD filesystem. 


Minix A filesystem that was the Linux standard a long time ago (it even predates Linux). It never hurts 
to include Minix filesystem support as a module. 


Pseudo filesystems These are system interfaces, not storage mechanisms. The /proc filesystem is 
one of the best-known pseudo-filesystems, and one that you should a/ways configure. You should also 
include the /dev/pts filesystem and the virtual memory filesystem. 


Network filesystems These are used for sharing files with machines over the network. 


There are also a couple important filesystem-related options: 


Kernel automounter support Enables automatic filesystem mounting and unmounting. This is a 
popular solution for managing network filesystem access. 


Partition types Allows Linux to read partitioning schemes other than the regular PC partition tables, 
including BSD disklabels and Solaris x86 partitions. 


This wraps up the important kernel configuration options. You shouldn't really have to worry about the 
other options (such as the Profiling support used for kernel development); let's turn our focus to compiling 
the kernel. 


10.4.2Compiling the Kernel and Modules 


After you save your kernel configuration, you're ready to build the kernel. To get a list of make targets, you 
can run this command: 


make help 


Your main goals are to compile the bzImage compressed kernel image and the kernel modules. Because 
these two are the default targets, you only need to run the following to build everything: 


make 


The compile takes some time. As it runs along, you see messages like this: 


CC inity Versi On.6 


If you see a message containing [M], it means that make is compiling a module: 


CC [M] net/sctp/protocol.o 


The following output appears when make builds bzImage: 


Kernel: arch/i386/boot/bzImage is ready 


As the preceding message indicates, the build process creates your kernel image as 
arch/i386/boot/bzImage. The process also creates a file named System.map that contains kernel 
symbols and locations. You may have to wait a little while after the "Kernel ready" message appears, 
becausemake may still need to compile some kernel modules. Don't interrupt the build; make sure that 
you wait until you get your prompt back before installing the kernel. 


Note In kernel versions prior to 2.6, you had to run the following commands for a complete kernel 
build: 


make dep 
make bzImage 


Failed Compiles 


Recent kernels are very self-contained; there are only two primary things that can go wrong during the 
build process: 


= |f you get a parse error or some other sort of coherent compiler error, the compiler on your machine 
probably doesn't match the recommended compiler in the README file. Kernel code is very particular 
about the compiler, especially due to all of the assembly code involved. 


a |f the compiler dies unexpectedly midway through the build process, your hardware may be at fault. 
Compiling a kernel stresses a machine much more than almost any other task. Bad memory and 
overclocked or overheated processors are the main culprits. 


lf you need to see each command in the build process to track down a problem, use this command: 


make V=l1 


10.4.3lnstalling Modules 


Before you install and boot from your new kernel, you should put the new kernel modules in 
/lib/modules with this command: 


make modules_install 


Your new modules should appear in /1ib/modules/version, where version is your kernel version. If 


you fail to install the modules before booting your kernel, the kernel module utilities will not automatically 
recognize the new modules, and you may end up missing a few drivers. 


The module installation builds a module dependency list in /lib/modules/version/modules.dep. If 
this stage fails, it's likely that your module utilities are out of date. As of this writing, the module utilities are 
ina package named module-init-tools. 


10.4.4Kernel Parameters 


Sometimes you need to send extra parameters to the kernel at boot time in order to get devices or 
services working. For example, one of the most elementary kernel parameters is root=partition, 


which sets the initial root partition to partition. 


You can enter a kernel parameter after the kernel image name. For example, at a LILO prompt where your 
Kernel label is Linux, you can type this: 


Linux root=/dey/ ndas 


Documentation/kernel-parameters.txt contains a list of all kernel parameters. Most of them are 
for hardware. In addition to root=partition, here are the most important parameters: 


init=path This starts path as the first process on the system in place of /sbin/init. For 
example, you can use init=/bin/sh to get out of a tight spot if your init does not work properly. 


mem=size This specifies that the machine has size memory; the kernel should not autodetect the 
memory size. For example, to specify 512MB of memory, you would use mem=512M. 


rootfstype=type This specifies that the root filesystem's type is type. 


A number as a boot parameter indicates the init runlevel. You can use -s or s for single-user mode. 


10.5Installing Your Kernel with a Boot Loader 


Boot loaders load kernel images into memory and hand control of the CPU over to the newly loaded 
kernel. To make your new kernel work, you must tell the boot loader about the kernel. 


If you have never worked with a boot loader before, you need to know how a PC boots from the hard disk. 
Ona normal Microsoft-based system, the boot process works like this: 


1. After performing the power-on self test (POST), the BIOS loads sector 1 of the hard disk into 
memory and runs whatever landed in that memory. On most Windows-based systems, this sector is 
called the Master Boot Record (MBR). 


2. The MBR extracts the partition table from the disk and identifies the "active" partition. 


3. The MBR loads and runs yet another boot sector from the active partition, setting the operating 
system in motion. 


On a Linux system, you can install the kernel boot loader on the active partition or you can replace the 
MBR with the boot loader. If you decide to replace the MBR with a boot loader such as GRUB or LILO, itis 
important to remember that the active partition has little meaning — these boot loaders address partitions 
based on their own configuration systems. 


From a practical point of view, your decision to overwrite the MBR (and thus circumvent the active 
partition) makes little difference. If your machine runs Linux, you probably want GRUB or LILO as your 
default boot loader. Even if you have a dual-boot machine, you still want to use GRUB or LILO, because 
both boot loaders are capable of loading boot sectors from other operating systems. 


The important point is that you know exactly where your boot loader resides. Let's say that your disk is 
/dev/hda and you have a Linux root partition at /dev/hda3. If you replace the MBR by writing GRUB to 
/dev/hda, you need to remember that the active partition is now irrelevant; GRUB uses its own 
configuration system to access partitions. However, if you decide to write the boot loader to /dev/hda3 
instead, keeping the old MBR, your system will get to that boot loader only if /dev/hda3 is the active 
partition. 


Note If you need to replace the MBR on a hard disk, run the DOS command FDISK /MBR. 


When configuring a boot loader, be sure that you know the location of the root partition and any kernel 
parameters. 


10.5.1Which Boot Loader? 


Because there are two popular Linux boot loaders, you may wonder which one you should choose: 


LILO Linux Loader. This was one of the very first Linux boot loaders. Its disadvantages are that it is 
not terribly flexible and that you must run 1110 to install a new boot block every time you install a new 
kernel. However, LILO is fairly self-contained. 


GRUB Grand Unified Boot Loader. This is a newer boot system gaining in popularity, and it does not 
need a reconfiguration for every new kernel because it can read many kinds of filesystems. This 
feature is especially handy for situations where you might need to boot from an old kernel. Initially, 
GRUB is slightly trickier to configure than LILO, but it is much easier to deal with once installed, 
because you do not need to replace the boot sector for every new kernel. | recommend GRUB. 


10.5.2GRUB 


With GRUB, you need only install the boot loader code on your machine once; after that, you can modify a 
GRUB menu file for any new kernel that you want to boot on a permanent basis. 


The GRUB boot files are in /boot /grub. GRUB loads the various stage files into memory during the 


boot process. If you already have GRUB on your machine, you just need to modify the 
/boot/grub/menu. 1st file when you add a new kernel image under a new name. (If you install a new 


kernel image with the same name as the old kernel, you do not need to modify the menu file.) 


Some distributions preinstall GRUB but have a different name for menu.lst . You may need to dig 


around in /boot to find the correct filename. In any case, the menu file looks like this: 


default o0 
timeout 10 


title Linux 
kernel (hd0,0)/boot/vmlinuz root=/dev/hdal 


title backup 
kernel (nd0,0) /boot/vmlinuz.0: root=/dev/hdal 


The parameters in the menu file are as follows: 


default The title entry that GRUB should boot by default. 0 is the first entry, 1 is the second, and 
so on. 


timeout How long GRUB should wait before loading the default image. 
title A label for a kernel. 


kernel The kernel image and parameters, possibly including the root device. 


In GRUB, (hao) is the first hard disk on your system, usually /dev/hda. GRUB assigns disk mappings in 
the order that they appear on your system, so if you have /dev/hda and /dev/hdc devices but no 
/dev/hdb, GRUB would assign (hd0)to/dev/hda and (hd1) to /dev/hdc. The first number in the 
GRUB name is for the disk, and the second is for the partition (if there is a partition). Check the 
/boot/grub/device.map file for your system's hard drive mapping. 


The kernel images in the preceding example are on the root partition of the primary master ATA hard disk. 
There are two kernel images: a regular kernel at /boot /vmlinuz anda backup at /boot/vmlinuz.0. 


Root Device Confusion 


You may find it odd that the root partition is actually specified twice in the kernel line in the preceding 
example (you can see (hd0,0) and /dev/hdal). This is easy to explain: the (hd0, 0) is where GRUB 
expects to find the kernel image, and root=/dev/hdal is a Linux kernel parameter that tells the kernel 
what it should mount as the root partition. These two locations are usually, but not always, the same. 
However, the GRUB and Linux device names are completely different, and so you need them in two 
places in the configuration file. 


Unfortunately, you may see this alternate syntax in menu.lst: 


root (hdu,.0) 
kernel. /boot/vmlinūüz.0 root=/dev/hdal 


This is confusing, because GRUB does not pass its root parameter (hd0, 0) to the Linux kernel. 
Omitting the GRUB root parameter can prevent some head scratching. 


Booting Other Operating Systems 


There are all sorts of other things you can do with GRUB, like load splash screens and change the title 
(uSeinfo grub to see all of the options). However, the only other essential thing you should know is how 
to make a dual-boot system. 


Here is a definition fora DOS or Windows system on /dev/hda3: 


title dos 
rootnoverify (hd0,2Z) 
makeactive 
Chainloader +1 


Remember how the PC boot loaders usually work, by first loading whatever is on the first sector of the 
disk, then loading the first sector of the active partition. 


Installing GRUB 


To put GRUB on your system for the first time, you must make sure that you have a menu. 1st file. The 
GRUB installation process does not create menu. 1st; you must come up with this file on your own. If you 


don't, you can still boot your system, but you must type a series of commands resembling entries in 
menu.lst todo so, as follows: 


kernel (hd0,0)/boot/vmlinuz root=/dev/hdal 
boot 


To install the GRUB software, run this command to put the boot sector on your disk: 


grub-installdevice 


Here, device is your boot device, such as /dev/hda. However, if you have aspecial /boot partition, you 
need to do something like this: 


grub-install --root-directory=/boot device 


After running grub-install, review your /boot /grub/device.map file to make sure that the devices 
relevant to booting the kernel are in the map file and agree with your menu. 1st file. 


10.5.3LILO 


To load your kernel with LILO, you must settle on a permanent location for your kernel image and install a 
new boot block for every change you make to the kernel configuration. 


Let's say that you want to boot /vmlinuz as your kernel with a root partition of /dev/hdal. Furthermore, 
you want to install the boot loader on /dev/hda (replacing the MBR). Do the following: 


1. Move the new image into place at /boot/vmlinuz. 


2. Create the LILO configuration file, /etc/lilo.conf. An elementary configuration might look like 
this: 


boot=/dev/hda 
root=/dev/hdal 
install=/boo0r/boot.b 
map=/boot/map 
vga=normal 

delay=20 


image=/boot/vmlinuz 
label=Linux 
read-only 


3. Runlilo -t -v to test the configuration without changing the system. 
4. Runlilo to install the boot loader code at /dev/hda. 


You can add more images and boot sectors to the LILO configuration. For example, if you saved your 
previous kernel as /boot /vmlinuz.0, you could add this to your lilo.conf to make the old image 
available as backup at the LILO prompt: 


image=/boot/vmlinuz.0 
label=backup 
read-only 


You 


can use the other keyword for foreign operating systems. The following addition to Lilo.conf 


offers a Windows partition on /dev/hda3 as dos at the LILO prompt: 


other=/dev/hda3 


label=dos 


LILO Parameters 


Some of the most important 1i1o.conf parameters are listed here: 


10. 


boot=bootdev Writes the new boot sector to the bootdev device. 
root=rootdev Uses rootdev as the kernel's default root partition. 


read-only Mounts the root partition initially as read-only. You should include this for normal boot 
situations (init remounts the root partition in read-write mode later). 


append="options" Includes options as kernel boot parameters. 


delay=num Displays the LILO prompt for num tenths of a second at boot time before booting the 
default kernel image. 


map=map_file Stores the kernel's location in map_file. Don't delete this file. 
install=file Specifies that file isthe actual boot loader code that 1i10 writes to the boot sector. 


image=file Defines a bootable kernel image with file. You should always use the label 
parameter directly following this definition to name the image. 


label=name Uses name to label the current boot entry. 


other=partition Defines another partition that contains a boot sector; analogous to image for 
other operating systems. Use label to define a label for the boot sector. 


linear Remaps the boot sector load references when the BIOS disk geometry is different than the 
geometry that the Linux kernel sees. This is usually not necessary, because most modern BIOS code 
can recognize very large disks. 


9.4Initial RAM Disks 


Aninitial RAM disk is a temporary root partition that the kernel should mount before doing anything else. 


Red 
com 


Hat Linux uses initial RAM disks to support SCSI host controller drivers and other drivers that are 
piled as modules. 


Note You do not need an initial RAM disk if you compile all the drivers necessary to mount your root 


filesystem directly into your kernel. The overwhelming majority of systems do not need an initial 
RAM disk. 


To install an initial RAM disk on a Red Hat Linux system, follow these steps: 


1. 


Build your kernel and install the modules. Do not run any boot loader configuration or reboot just 
yet. 


2. Run this command to create a RAM disk image (where version is your kernel version): 


mkinitrd /boot/initrd-version version 


3. If your boot loader is LILO, add this line to your new kernel's section in 1ilo.conf andrun lilo: 


initrd=/boot/initrd-version 


4. For GRUB, add the following to the appropriate kernel section: 


initrd /boot/initrd- version 


10.6Testing the Kernel 


When you boot your new kernel for the first time, you should take a careful look at the kernel diagnostic 
messages to make sure that all of your hardware shows up and that the drivers are doing what they are 
supposed to do. Unfortunately, kernel messages tend to fly by so quickly that you can't see them. You can 
rundmesg to see the most recent messages, but to see everything, you need to look at your log files. 


Most/etc/syslog.conf files send kernel messages to a file suchas /var/log/kernel.1log. If you 
don't see it anywhere, add a line like this to your /etc/syslog.conf: 


kern.* /var/log/kernel.log 


Then run this command: 


kill -HUP '/var/run/syslogd.pid' 


You may wish to make a checklist for your new kernel to make sure that your system still operates as it 
should. Here's an example: 


=» Do the network interface and network firewalls work? 

a Are all of your disk partitions still visible? 

= Does the kernel see your serial, parallel, and USB ports? 
=» Does all of your external hardware work? 


=» Does the X Window System work? 


10.7Boot Floppies 


You can also boot the kernel from a floppy disk. Creating a boot floppy can be useful when recovering a 
Linux system that has no active kernel. 


To create a boot floppy, put a freshly formatted floppy disk in the drive, go to the kernel source directory, 
and run this command: 


make bzdisk 


Of course, this only works if the size of your bzImage is smaller than the floppy disk capacity. 


You may also need to run rdev to set the floppy's default root device. For example, if your root partition is 
/dev/hdal, use this command: 


rdev /dev/td0 /dev/hdal 


Again, boot floppies are primarily useful during a system recovery when there is no active kernel. For 
testing new kernels, it's far better just to use an advanced boot loader such as GRUB. 


10.8Working with Loadable Kernel Modules 


Loadable modules are little pieces of kernel code that you can load and unload into the kernel memory 
space while the kernel is running. 


Themake modules_install command (discussed in the "Installing Modules" section) installs the kernel 
module object files in /lib/modules/ver-sion, where version is your kernel version number. Module 
object filenames end with . ko in kernel versions 2.6.0 and later, and .o in older releases. 


All distributions use modules in some capacity. If you would like to see the modules currently loaded on 
your system, run this command: 


lsmod 


The output should look something like this: 


Module Size Used by 
es1370 24768 O0 (autoclean) 
appletalk 19696 13 (autoclean) 


This output includes es1370 (a sound card driver) and appletalk (a network protocol driver). 
autoclean means that the kernel may try to automatically unload the module if it is not used for some 
time. 


To load a module, use the modprobe command: 


modprobemodule 


To remove a single module, use the -r option: 


modprobe -r module 


As mentioned earlier, you can find module dependencies in /lib/modules/version/modules.dep. 
Dependencies dont arrive on your system by magic; you must build an explicit list (the kernel module 
install process usually does this for you). You may need to create module dependencies by hand for all 
installed kernel versions after installing a module that doesn't come with the kernel. You can do this by 
running this command: 


depmod -a 


However, this doesn't always work, because depmod may try to read the symbol function memory address 
locations in the currently running kernel. This won't work if you're trying to build dependencies for kernels 
other than the one you're running. To zero in on a particular kernel version, find a System.map file that 
corresponds to that kernel, and then run this command: 


depmod -a =F System.map version 


As mentioned earlier, though, you do not need to run depmod under normal circumstances, because make 
modules_install runs it for you. 


HINT If you can't find the module that corresponds to a particular feature, go to the feature in the 
kernel configuration menu and press ? to get the help screen. This usually displays the module 
name. 


10.8.1Kernel Module Loader 


It's inconvenient to manually load a module every time you need to use a particular feature or driver. For 
example, if you compiled a certain filesystem as a module, it would be too much work to run a modprobe 
before a mount command that referenced the filesystem. 


The Linux kernel provides an automatic module loader that can load most modules without additional 
modprobe or insmod commands. To use the loader, enable the Kernel module loader option in the 
kernel configuration. A kernel with the module loader runs modprobe to load modules that it wants. 


There are limits to what the module loader can do without additional configuration. In general, it can load 
most modules that do not involve specific devices. For example, the module loader can load IP tables 
modules, and it can load filesystem modules as long as you specify the filesystem type with a mount 
command. 


The module loader cannot guess your hardware. For instance, the module loader will not try to figure out 
what kind of Ethernet card is in your system. Therefore, you need to provide extra hints with the 
modprobe.conf file. 


10.8.2modprobe Configuration 


Themodprobe program reads /etc/modprobe.conf for important device information. Most entries are 
aliases such as this: 


alias ethO tulip 


Here, the kernel loads the tulip module if you configure the ethno network interface. In other cases, you 
may need to specify drivers by their major device numbers, such as this entry for an Adaptec host 
controller: 


alias block-major-8 aic7xxx 


Wildcards are also possible in modprobe.conf aliases. For example, if all of your Ethernet interface 
cards use the tulip driver, you can use this line to catch all interfaces: 


alias eth* tulip 


Note The module utilities discussed here are the ones that go with kernel version 2.6.0. These 
programs are part of the module-init-utils package. Earlier kernel versions used the 
modutils package. The most significant difference between the two sets of utilities is that the 
new package reads modprobe. conf instead of modules.conf. The syntax in both files is very 
similar. 


Chaining Modules and the Install Keyword 


You can chain modules together with the install keyword. For example, if SCSI disk support isn't 
compiled directly into the kernel, you can force it, as in this example for /dev/sda on the Adaptec host 
controller from the preceding section: 


alias block-major-8 my_scsi_modules 
install my_scsi_modules /sbin/modprobe sd_mod; /sbin/modprobe aic7xxx 


This works as follows: 


1. A process (or the kernel) tries to access /dev/sda. Assume that this device is not yet mapped to 
an actual device. 


2. The kernel sees that /dev/sda isn't mapped to its device, which has a block major number of 8. 
Therefore, the kernel runs this command: 


modprobe block-major-8 


3. modprobe searches though /etc/modprobe. conf for block-major-8 and finds the alias 
line. 


4. Thealias line says to look for my_scsi_modules, SO modprobe runs itself, this time as follows: 


modprobe my_scsi_modules 


5. The new modprobe sees install my_scsi_modules IN modprobe.conf, and then runs the 
command(s) that follow in the file. In this case, these commands are two additional modprobe 
commands. 


You can include any command that you like in an install line. If you need to debug something or just 
want to experiment, try adding an echo command. 


Note There is a remove keyword that works like install but runs its command when you remove a 
module. 


Module Options 


Kernel modules can take various parameters with the options keyword, as shown in this example for a 
SoundBlaster 16: 


alias snd-card-0 snd-sb16 
options snd-sb1l6 port=0x220 irq=5 dma8=1 dmal6=5 mpu port=0x330 


Chapter 11: Configuring and Manipulating Peripheral 
Devices 


This chapter covers various peripheral devices that may be attached to your computer. Although compiling 
and installing the low-level drivers for your kernel are the first steps in getting any device including 
peripherals working, you may also need additional utilities or servers to do anything useful with the device. 
A great deal of the work involved when configuring peripherals is digging through kernel log messages to 
see exactly how the kernel assigns a device and whether the kernel recognizes the device correctly in the 


first place. Therefore, this chapter contains a great number of example kernel messages that you can use 
as a guide for your own installation. 


11.1Floppy Drives 


When working with a PC floppy drive, you must remember that it is one of the most unreliable pieces of 
hardware that you can find on aPC. You can mount the first floppy drive using the device /dev/ fao, but 
this can be dangerous, because the kernel will not like any hardware errors. Using the mtools programs is 
a good alternative to mounting the filesystem directly. 


The mtools programs look like their MS-DOS counterpart commands with an m at the beginning. For 
example, you can use mdir for a directory listing. 


To copy a file from a floppy disk to the current directory, run this command: 


mcopy a:file . 


Copying files from the local system to the disk is similar: 


mcopyfile a: 


You can also format a floppy with an MS-DOS filesystem using the mformat Command: 


mformat a: 


For more options and commands, refer to the mtools(3) manual page. 


11.1.1Floppy Images 


Sometimes it's more convenient to copy the entire image of a floppy disk to a file on your hard disk than it 
is to deal with individual floppy disk operations. You can perform the extraction with dd: 


dd if=/dev/fd0 of=image_file bs=512 


You might be able to make the extraction go a little faster by changing the bs parameter to 32k. After you 
have the image file, it's convenient to mount the image file on your system with a loopback block device. 
This example mounts an image of an MS-DOS filesystem: 


mount -t vfat -oloop=/dev/loop0 image_file mount_point 


Warning Remember to unmount the image file after you finish, especially if you plan to alter the 
filesystem in the image. 


11.1.2Low-Level Formatting a Floppy Disk 


If you absolutely need to write to a floppy disk, but you keep getting I/O errors, you can try to format a disk 
with the superformat program (which is part of the fdutils package) or with fdformat. 


11.2CD Writers 


Linux CD burning programs work with ATAPI devices and generic SCSI devices, such as /dev/hdc and 
/dev/sg0. The most popular command-line CD recording utility for Linux is named cdrecord. 


Most CD burners on the market are ATAPI devices. At one time, Linux supported these devices with an 


IDE-to-SCSI emulation driver called ide-scsi. However, with Linux kernel version 2.6, this is no longer 
necessary, and the ide-scsi driver may not work well in newer kernel releases. 


11.2.1Verifying the Hardware 


Before trying to write a CD, you should verify that the CD recorder drive exists and determine its device 
name and specification. For IDE devices, locating the device is easy; just use the standard device name 
for the entire disk (for example, /dev/hdc; see Section 2.3.4 for more information). 


You can verify the existence and identity of a device with the -inq option to cdrecord. For example: 


cdrecord -inq dev=device 


If there is an ATAPI device at device, the output of the cdrecord -inq command looks like this: 


Device type : Removable CD-ROM 
Version e U 

Response Format: 1 

Vendor_info ; 'PLEXTOR ' 
Identifikation : 'CD-R PX-W4012A' 
Revision > * 01" 


Device seems to be: Generic mmc CD-RW. 


You can search for a SCSI CD recorder device with this command: 


cdrecord -scanbus 


The output should look something like the following: 


scosibus0 : 


+ + + 


* 


"HP eee CASEY) ' '9503' Removable Tape 
"PLEXTOR ' "CD-R PX-R412C ' '1.04' Removable CD-ROM 


* 


* 


oOoOO0OO OO O 
NTO UB WN FE O 
oOoo0o0Oo0OoĦ0OOO O 
HUAN OP WN EF OO 


A Linux SCSI device has a bus (for the host controller), a target, and a LUN (Logical Unit Number). Take a 
close look at the bold terms in the preceding output; these are the bus, target, and LUN numbers. You can 
see that the SCSI bus is 0, the target of the burner (a Plextor drive) is 5, and the LUN is 0. Therefore, the 
device you need to know is 0,5,0. Most burner devices do not have multiple units, so you can abbreviate 
the device to 0,5. 


If you know the generic SCSI /dev/sg* device name of your CD-ROM, you can use that in place of the 
three-part numeric identifier. Look for messages like the following in your kernel logs to identify the generic 
device name. In this example, the device is /dev/sgL: 


Vendor: PLEXTOR Model: CD-R PX-R412C Rev: 1.04 

Type: CD-ROM ANSI SCSI revision: 02 
sr0: scsi3-mmc drive: 12x/12x writer cd/rw xa/form2 cdda caddy 
Attached scsi CD-ROM sr0O at scsi0, channel 0, id 5, lun 0O 


Attached scsi generic sgl at scsi0, channel 0, id 5, lun 0, type 5 


11.2.2Creating a Filesystem Image and Writing a CD 


To burn a CD, you need to create a new image of the filesystem to burn, then write that image to the CD 
writer. You can do this either with a temporary disk image file or in a pipeline. 


Using a Temporary Disk Image 
ISO9660 is the original standard CD-ROM filesystem format that most CD-ROMs use. 


To create an ISO9660 filesystem image from a directory on a Linux system, use mkisofs, like this: 


mkisofs -r -o image.isodirectory 


The-r option enables the Rock Ridge extensions that transform the stark ISO9660 filesystem into one 
that looks more like a Unix filesystem. You should use this for any CD-ROM that you intend for use on 
Unix systems. (For Microsoft systems, you may want to consider using —J for the Joliet extensions.) 


You can write an image to the burner with cdrecord: 


cdrecord -v dev=device speed=speed image.1so 
You can test a cdrecord command with the -dummy argument; this performs a recording run with the 


burning laser off. 


Before writing an image to a CD, you may also wish to test it with the loopback device (just remember to 
unmount it when finished!): 


mount =i =t 1s09660 -oloop=/dev/loop0O image.iso /mnt 


Using a Pipeline 


If you don't want to create a temporary image file, you can attempt to create the filesystem and write to the 
CD device in one pipeline: 


mkisofs -r directory | cdrecord -v dev=device speed=speed - 


The only problem you may encounter with this is a buffer underrun, where mkisofs can't produce data 
quickly enough for cdrecord to consume. If you run into a problem like this, one possible remedy is to 
lowerspeed, or you can see if there are any other processes on your system that are hogging the CPU. 


11.3lIntroduction to USB 


Linux supports a wide variety of USB (universal serial bus) devices. A USB system on a PC consists of a 
host controller inside your computer, hubs that act as splitters to give you more ports, and the actual USB 
devices. The Linux kernel supports USB devices through three types of drivers: 


= Host interface drivers 
=» USB device drivers 


a Other drivers not necessarily related to USB, but required by a USB device driver (for example, the 
USB mass storage driver requires the SCSI disk support driver) 


When the host interface driver recognizes a controller, the kernel logs messages such as these: 


ohci. hed: 2003. Oct 13.USB 1.1 “OGpen’ Host Controller (ORCI) Driver (PCI) 
ohci_hcd 0000:00:02.0: irag 9, pci mem d4806000 

ohci_hcd 0000:00:02.0: new USB bus registered, assigned bus number 1 
hub 1-071077 USB hub found 

hub 1-0:1.0: 2 ports detected 


Notice that the controller has an integrated hub with two ports. Plugging in external hubs yields kernel 
messages like the following: 


hub 1-0:1.0: new USB device on port 2, assigned address 3 
hub L-2si.02 USB hub found 
hub 1-2:1.0: 4 ports detected 


Note Don't be surprised if you see hub-related log messages when plugging ina USB device, even if 
the device is not a hub. Some of these devices contain hubs. 


When you plug in a USB device, the kernel asks the various device drivers if they know how to talk to the 
device. If a driver recognizes the device, it can claim the device and take over its management. 


11.3.1USB Device Filesystem and Utilities 


With the usbdevfs filesystem mounted at /proc/bus/usb on your system, you can take a look around 
this directory to see the details of the devices that are connected to your system. However, you can also 
get a short Summary with the 1susb command from the usbutils package: 


Bus 001 Device 005: ID 054c:002d Sony Corp. MSAC-US1 Memorystick Reader 
Bus 001 Device 003: ID 0451:2046 Texas Instruments TUSB2046 Hub 

Bus 001 Device 002: ID O046d:c00c Logitech Inc. Optical Wheel Mouse 

Bus 001 Device O01: ID 0000:0000 Virtual Hub 


If you want to see the gory details of all of your devices, run lsusb -v. Be warned, though, that this prints 
an extremely verbose listing. 


11.3.2USB Input Devices 


Basic USB keyboard and mouse support is not complicated. To use these devices on your system, you 
need the USB human input device (HID) driver in your kernel. The relevant kernel log message looks like 
this: 


drivers/usb/input/hid-core.c: v2.0:USB HID core driver 


With this in place, any USB keyboard that you plug into your computer should work with no further 
configuration. Here is the message that the kernel logs when you plug in an old Apple keyboard: 


input: USB HID vl.00 Keyboard [Alps Electric Apple USB Keyboard] on 
Gseb=0000200.02..0=2..1 


USB mice aren't much more difficult to configure; the only thing you need to do is to tell the X server or 
gpm where to find a USB mouse (look at your /etc/X11/XF86Config file and grep for gpm in your 
init .d directory). The kernel maps all mice to /dev/input /mice, using the PS/2 mouse driver. The 
kernel message that results from the hookup of a new mouse looks like this: 


input: USB HID vl.10 Mouse [Logitech USB Optical Mouse] on usb-0000:00:02.0-1 


11.3.3USB Digital Cameras, External Flash Media, and External 
Disks 


Linux accesses external USB storage devices, including flash memory, in blocks, presenting the storage 
as a SCSI disk. Therefore, you need the following drivers in your kernel if you would like to use such a 
device: 


a» SCSI disk support, compiled directly into your kernel or as the sd_mod kernel module 
=» A USB interface driver (see Section 11.3) 
= USB mass storage support, compiled directly into your kernel or as the usb-storage module 


Assuming that you have all of the drivers in place, you should see a kernel log message proclaiming that 
there is anew SCSI bus when you plug in a flash media card device or other external mass storage 
device: 


scsil : SCSI emulation for USB Mass Storage devices 
Vendor: Sony Model: MSAC-US1 Rev: 1.00 
Type: Direct-—Access ANSI SCSI revision: 02 


SCSI device sda: 15840 512-byte hdwr sectors (8 MB) 
sda: Write Protect is off 
sda: Mode Sense: 00 06 00 OO 
sda: assuming drive cache: write through 
sda: sdal 
Attached scsi removable disk sda at scsil, channel 0, id O0, lun O 
Attached scsi generic sg2 at scsil, channel O, id U; lun U; type O 


Upon disconnecting the device, you will see a message like this: 


usb 1-2.1: USB disconnect, address 4 


Seeing the new SCSI bus in the kernel log does not necessarily mean that you have access to the 
attached USB device. If your system does not have hotplug support, you may need to run a script called 
rescan-scsi-bus.sh to get the kernel to recognize the new device. You can verify the geometry, 
partitions, and filesystems on the storage device with an fdisk command such as this: 


fdisk -1 /dev/sda 


Flash Memory Filesystem Issues 


The flash memory in most digital cameras contains an MS-DOS filesystem that you can easily mount 
when you see the device in the kernel logs. 


Warning You may wish to mount the filesystem in read-only mode, because it is too easy to remove 
flash media cards and devices before unmounting the filesystem. If you alter the filesystem, 
then remove the card before flushing the changes to the physical media with an unmount 
operation, you risk data corruption. 


11.41EEE 1394/FireWire Disks 


IEEE 1394 (also known as FireWire) drivers are relatively new to Linux. The Linux IEEE 1394 driver system is 
similar to that of the USB system, and the Linux kernel support for IEEE 1394 includes the base module 
(i1eee1394) and host interface drivers. One common interface is OHCI-1394 (which has a module name of 
ohcil394), and the TI PClLynx interface also has a driver (pcilynx). 


Kernel messages like the following indicate that the host interface driver detected the hardware: 


oheil394_0: OHCI-1394 1.1 (PCI): IRQ=[26] MMIO=[9006000-90067ff] Max Packet=[2048] 
1eeel394: Host added: ID:BUS[0-00:1023] GUID[0001080020005cb4] 


IEEE 1394 hard disks use SBP-2 (Serial Bus Protocol). As in the USB mass storage code, the Linux sbp2 
driver masquerades as a SCSI host controller. When you plug in a drive, look for these kernel messages: 


Sscs10 ¢ SCSI emulation for IREE=1394 SBP=-2 Devices 

1eeel394: sbp2: Logged into SBP-2 device 

1eeel394: sbp2: Node 0-02:1023: Max speed [S400] - Max payload [2048] 
1eeel394: Node added: ID:BUS[0-02:1023] GUID[0090a95000001796] 


If your system does not have hotplug support (See Section 11.5), you may need to probe the SCSI bus with 
therescan-scsi-bus.sh script described earlier in order to get the kernel to recognize the IEEE 1394 
drive. Upon recognizing the drive, the kernel emits the usual SCSI drive messages: 


Vendor: WDC Model: FireWire/USB2.0 Rev: 4.17 
Type: Direct—-Access ANSI SCSI revision: 06 

blke queue ffrrro001954p928; no 1/70 memory Limit 

Attached scsi disk sda at scsi0, channel 0, id O, lun O 

SCSI device sda: 234441648 512-byte hdwr sectors (120034 MB) 
sda: sdal sda2 


Note The Apple iPod is an SBP-2 device. To access the disk and its filesystem, you need appropriate 
filesystem support and possibly also the appropriate partition table support in your kernel. The PC 
version of the iPod uses the FAT 32 filesystem, so you can use the vfat filesystem. The Macintosh 
version uses a completely different filesystem type, so you need the Macintosh partition map support 
in your kernel, as well as Macintosh filesystem (HFS) support as a module or compiled directly into 
the kernel. 


11.5Hotplug Support 


Now that we've covered Linux USB and IEEE 1394 support, you should learn a little about Linux hotplug 
support, a system that can handle the automatic configuration and cleanup of devices when you plug them 
into the system. 
The idea is simple; the kernel runs /sbin/hotplug when you plug in a device. Here's the procedure: 

1. You plug in a device. The kernel notices. 

2. The kernel runs /sbin/hotplugtype, where type is the device type (usb,ieee1394,scsi, 


and so on). The kernel sets a number of device-related environment variables in the new hotplug 
process. 


3. The/etc/hotplug script runs /etc/hotplug/type.agent. 
4. The/etc/hotplug/type.agent script tries to configure the device. 


There really isn't much more to it than this. Each device type's . agent file has its own method of 
configuring devices. For example, usb. agent goes through these steps: 
1. usb.agent looks In /etc/hotplug/usb.distmap and /etc/hotplug/usb.usermap to 
locate a kernel module for the devices. 


2. Having identified a module, usb. agent attempts to load the module. 


3. If/etc/hotplug/usb/type exists, usb. agent runs that program. 


Linux hotplug is almost entirely script-driven, which means that the details are even more subject to 
change than the rest of Linux. However, keep in mind that if you ever have problems or questions you can 
always read the hotplug scripts to determine what they are trying to do, even if the documentation is 
inadequate or out of date. 


11.6PC Cards (PCMCIA) 


Linux PCMCIA support is highly dependent on kernel versions. If you upgrade your kernel, you almost 
certainly need to upgrade your PCMCIA drivers and tools. 


Although some of the core PCMCIA drivers are in the main Linux kernel distribution, you still need the 
pcmcia-cs package for additional drivers and system utilities. The components of the PCMCIA system 
include the following: 


=» Kernel modules for PC Card interface controllers and individual PC Cards. These modules go in 
/lib/modules/version/pcmcia. 


a The/sbin/cardmgr daemon for managing configuration when you insert or remove a card. 
a Various support utilities and services, located in /etc/pcemcia and /sbin. 


There are too many pieces of the system to set up and configure by hand, so the pcmcia-cs package 
provides several scripts to get you on your way. The most important of these is an init.d script called 
pcmcia, where, depending on your distribution, you may need to modify a few lines to specify which 
PCMCIA interface controller chipset you have. Look at this section: 


# Source PCMCIA configuration, if available 
if | =£ /eteG/ocmeia.cont ] = then 
# Debian startup option file 
j/ete/pencia.cont 
elit | -f /etc/syscontig/pcmcia ] 4 then 
# Red Hat startup option file 
/etc/sysconfig/pcemcia 
else 
# Slackware startup options go right here: 
# Should be either 182365 or tcic 
PCIC=i82365 
# Put socket driver timing parameters here 
PCIC. OPTS= 
# Put pcmcia_core options here 
CORE _OPTS= 
# Put cardmgr options here 
CARDMGR_OPTS= 
# To set the PCMCIA scheme at startup... 
SCHEME= 


The mention of specific Linux distributions may seem confusing, so step back and recall your shell script 
essentials from Chapter 7. The script does the following checks and actions: 


1. If/etc/pomcia.conf exists, the script reads configuration details from there and skips past the 
fi. This is probably the case if you're running Debian GNU/Linux. 


2. If step 1 fails, the script looks for /etc/sysconfig/pcmcia, which is the location of the pcmcia 
file in Red Hat Linux. 


3. If steps 1 and 2 both fail, the script directly sets PCIC,PCIC_OPTS, and so on. Note that this may 
apply to distributions other than Slackware Linux. 


To get cardmgr talking to your PCMCIA interface controller, you should only need to set the PCIC 
variable. Check through your init.d and /etc files to find the file in which you need to set this variable 
(every distribution is different, so using grep PCIC may come in handy). If you don't Know what sort of 
hardware you have, there is a pcic_probe program that comes with the pcmcia-cs source code that 
can help you. The value of PCIC should match one of the module filenames in 
/lib/modules/version/pcmcia; it's typically something like 182365 ortcic. 


With the correct PCIC setting in place, you can manually load the interface controller driver and start 
cardmgr with this command: 


pcmcia start 


However, because pcmcia isan init.d script, you should create rc.d links to start and stop the 
PCMCIA services at boot and shutdown time (refer back to Section 3.1 if you're a little hazy on this 
process). 


11.6.1When You Insert a Card 


Thecardmgr program reads configuration data and activates scripts in /etc/pcmcia in response to 
cards being inserted and removed. The central card configuration database is /etc/pcmcia/config. 


When you insert a PC Card, cardmgr does the following: 
1. It reads the device identifier information from the PC Card. 


2. It looks up the identifier in /etc/pcmcia/config, retrieving the kernel module name that contains 
the device driver. 


3. It matches the kernel module name with a class name in /etc/pemcia/config. 


4. It loads the kernel module(s) and configures the PC Card with respect to the local options in 
/etc/pemcia/config.opts. 


5. It runs a class script located inthe /etc/pcmcia directory, corresponding to the class name it 
found in /etc/pemcia/config. 


/etc/pcmcia 


Let's look at the /etc/pcmcia directory in more detail. Each kernel module has one or more matching PC 
Card definitions in the /etc/pcmcia/config file. For example, here is an entry that matches the 
3c589_cs module against its vendor and version identifier strings: 


card "Lucent Technologies WaveLAN Adapter" 
version "Lucent Technologies", "WaveLAN/PCMCIA" 
bind "wavelan_cs" 


Thebind in the last line means to bind the card definition to one or more kernel modules. However, the 
card definition does not always specify that version of the hardware. cardmgr can also identify the 
appropriate kernel module by using other hardware criteria, including the function parameter, as in this 
example: 


card "ATA/IDE Fixed Disk" 
function faxed. disk 
bind "ide_cs" 


Well-defined device interfaces such as disks, serial ports (modems), and flash memory usually have a 
function parameter, but don't expect this from all devices. In particular, Ethernet cards usually don't 
have a function parameter. You may also see manual device-identifier matches that use the manfid 
keyword. 


Having identified the necessary kernel module(s), cardmgr then looks for the device driver module class, 
which are found in the first part of /etc/pcemcia/config. Here is a sample definition that maps 
wavelan_cs tothe network driver class: 


device "wavelan_cs" 
class "network" module "wavelan_cs" 


Each driver class has a script in /etc/pcemcia that cardmgr runs after loading and configuring a card's 
module (you'll see more about class scripts shortly). 


Configuration Options 


You should not modify /etc/pcmcia/config; instead, enter your local configuration data in 


/etc/pemcia/config.opts (cardmgr usually reads this file after config). For the most part, the 
PCMCIA utilities should work fine with the default settings, but the machine-and card-specific options in 
config.opts are at your disposal if your hardware just won't behave. 


Of particular interest are the exclude and include keywords that enable you to manually specify IRQs 
and addresses. For example, you will see the following lines to prevent cardmgr from using standard 
IRQs, regardless of what the kernel may say about their availability: 


# First built-in serial port 
exclude irg 4 


# First built-in parallel port 


exclude irg 7 


You can specify per-module options with the module keyword. These examples are also from the default 
config.opts file: 


module "ray_cs" opts "net_type=1l essid=ESSID1" 


module "wvlan_cs" opts "station_name=MY_PC" 


Note In normal circumstances, you do not configure wireless options with config.opts, but rather, 
withiwconfig and possibly /etc/pcmcia/wireless.opts (see Section 5.15 for more 
information). 


Auxiliary Configuration Scripts 


Each driver class has a script in /etc/pcmcia. For example, the /etc/pcmcia/ network script 
corresponds to the network class mentioned earlier. cardmgr runs the class script after loading and 
configuring a card's module. 


The/etc/pemcia configuration scripts work like init.d boot scripts, taking an argument to start, stop, 
reload, or perform other actions: 


start Activates the device upon card insertion 

stop Deactivates the device upon card removal 

restart Reinitializes the device 

check Verifies the device 

suspend Deactivates a card, but leaves the driver running 


resume Picks up at the point where suspend left off, if possible 


Just because a script exists in /et c/pcmcia doesn't mean that it actually works. These scripts run 
through several combinations of distribution-specific configuration files, trying to find the right settings. If 
your PC Card doesn't auto-configure correctly upon insertion, you may need to modify one of the scripts or 
its corresponding . opts configuration file. 


Warning You may find it easiest to write your own driver class scripts or modify the existing ones, but 
remember to make a backup of any script you decide to change. Reinstalling the pcmcia- 
cs package overwrites each script in /etc/pcmcia (leaving a backup with a .o extension). 
The. opts configuration files remain in place during a reinstall. 


You should not normally need to call one of the /etc/pcmcia scripts by hand. cardmgr and cardctl 
(discussed next) do this for you. 


11.6.2cardctl 


Thecardct1 utility enables you to alter card settings and display information about cards and sockets. 
For example, you can view PC Card identifiers with this command: 


runas cardctl ident 


The output looks similar to this: 


Socket Ü: 
product info: "Lucent Technologies", "WaveLAN/IEEE", "Version 01.01", "" 
manfid: Ox0156, Ox0002 
function: 6 (network) 


Notice how this information relates to the configuration data in /etc/pcmcia/config. 


Here are some other cardct 1 commands: 
config Shows detailed low-level hardware settings. 


status Displays card or socket status. 


ejectsocket Shuts down the card in socket (use ident to identify the socket number). You 
should run eject oncertain cards, such as SCSI cards and other interfaces that take a long time to 
initialize, before removing them. 


insertsocket Tells any relevant drivers that you just put a card into socket. Like eject, this is 
not necessary for most cards. 


suspendsocket Attempts to freeze the driver for socket and cut the power. 
resumesocket Attempts to recover from a suspend operation. 
reset socket Resets the cardin socket. 


There are a few more complex options for cards that need to operate in multiple environments (schemes); 
see the cardctl(8) manual page and the PCMCIA-HOWTO for more information. 


Note Mostpcmcia-cs builds also come with cardinfo, a GUI display of the PCMCIA configuration. 
You may prefer to run this application in place of information commands such as cardct1 
status. 


11.7Approaching Other Devices 


Obviously, the number of devices that you can attach to your PC is enormous, and attempting to write a 
chapter that covers all of them is next to impossible. However, with what you just learned about SCSI, 
USB, PCMCIA, and IEEE 1394, along with what you will learn about printers in Chapter 12, Ethernet 
interfaces, and so on, you should be well equipped to take on almost anything. For example, you know 
how to identify a USB scanner; it's just a matter of installing the appropriate drivers and software (the most 
popular Unix scanning system is called SANE, if you are interested). 


This book isn't finished with devices just yet, though. In particular, Chapter 13 has information about tape 
drives, and Chapter 17 contains tips on how to purchase hardware so that you can avoid Linux 
compatibility nightmares. 


Chapter 12: Printing 


Overview 


The printing process on Unix works roughly as follows: 


1. 


2 
3. 
4 


gi 


ig 


An application generates a document to print. 


. You or the application sends the document to a print server by means of a print client program. 


The print server puts the document on a queue. 


. When it is the document's turn to print, the print server sends the document through a print filter 


program. 
The print filter changes the document to make it acceptable to a printer. 


If the printer does not understand PostScript (the standard document format in Unix), another print 
filter must rasterize the document into a bitmap form that the printer understands. 


The print server sends the result to the printer device. 


Making your printer work on a Unix system can be a trying experience. There are several problems: 


There are few standards. 


Each stage of the process can present any number of errors. 


There are several different client, server, and filter packages for printing. 


It's sometimes difficult to determine the purpose of each printing system component. 


The existing software never seems quite satisfactory, so upgrades are frequent. 


Security holes run rampant in network print servers. 


There are two basic ways to configure network printing: one where all computers send their 
documents directly to a printer, and the other where all computers send their documents to a print 
server, which then talks to the printer. It can be confusing when you have a network full of machines 
that use both approaches. 


With all of this said, you may not even want to bother to learn how printing works. To be perfectly honest, 
if your distribution's printer setup tool works for you, it may not be worth going any further, as long as you 
know how to firewall the print server's network port. However, when things don't work according to plan, 
the information in this chapter will help you find out where the problem lies. 


This chapter first explains the pieces of the printing system on Linux and then shows you how to put 
everything together using CUPS (Common Unix Printing System). The best place to start is the print code 
that applications create, PostScript. 


12.1PostScript 


Unix applications that print generate PostScript files as output. PostScript is not a simple file format; 
rather, it is a page description language. Therefore, PostScript documents are programs, so you need an 
interpreter to view or read them. Most mid-to high-end laser printers understand PostScript, as do certain 
inkjet printers, but the cheapest printers do not. If you have such a printer (and who doesn't like to buy 
cheap hardware?), your computer must perform the extra step of rasterization before the document is sent 
to the printer. 


When you rasterize a PostScript file, you convert its vector graphics into a bitmap. On Linux, the 
Ghostscript program does this work. Ghostscript can produce output for almost any printer on the market, 
but it also has many other uses because it is a full-featured PostScript interpreter. See Section 12.6 for 
more information. 


If you are interested in PostScript as a programming language, PostScript Language Tutorial and 
Cookbook [Adobe 1985] is a good introduction to the language, and PostScript Language Reference 
Manual [Adobe 1999] serves as a comprehensive guide to the language features. However, if you just 
want an example PostScript file for tests, use a Web browser's print-to-file feature. 


12.2Print Servers 


There are several different Unix print servers. Here are just a few: 


Berkeley LPD (Line Printer Daemon) One of the oldest print servers around, this system is fairly 
stable, but is somewhat sparse on features and can be difficult to configure. Many network-capable 
printers have integrated LPD servers. 


LPRng This new spin on LPD has many new features, but in all honesty, is not any easier to 
configure. You may also see a related package named PLP (Portable Line Printer). 


lpsched This is the System V printing system. If you're lucky, you'll never have to see it. 


CUPS (Common Unix Printing System) Sooner or later, someone was bound to get so fed up with 
the ugly state of printer clients and daemons on Unix that they wanted to rip everything to pieces. 
CUPS is the latest entry in the Unix printing field, and it may prove to be the most successful. Its 
internals and protocol departs from the older printing systems, but it still has a server (called the 
scheduler) and print filters. In addition, the client commands are mostly identical to the lpsched 
commands, with LPD-compatible substitutes as well. 


This chapter concentrates on CUPS, which is quickly gaining popularity because it is definitely much 
easier to configure and use than the alternatives. In particular, CUPS offers a Web-based interface for 
many of the more mundane printing tasks. 


12.3Print Filters 


Two print filters dominate Unix printing: 


Foomatic This print filter works with almost any print server, and on the command line. Foomatic can 
call Ghostscript for printers that do not understand PostScript. 


ifhp This is a general-purpose filter for PostScript printers. ifhp is an older print filter that works well 
with the Berkeley LPD-style servers. 


12.4Print Clients 
There are two traditional clients that send data to a print server: 


lpr This is the matching client for Berkeley LPD-based servers. Associated programs include 1pq (to 
view the current printer job queue) and 1prm (to remove a job from the queue). 


lp Both CUPS and System V Ipsched use the 1p interface to print (CUPS also has limited lpr 
capability). The 1pstat command shows printer jobs, and cancel removes a print job. 


On traditional Linux systems, you send a PostScript job to the printer named printer with a command 
like this (you do not need -Pprinter if you want to send the job to the default printer): 


lpr -Pprinter file 


For CUPS, you may also use the 1p program (-—dprinter is optional if you are sending to the default 
printer): 


lp -d printer file 


Commands like lpr and 1p let you send any PostScript file or data pipeline directly to a printer. However, 
one of the biggest problems with this model is that users have no idea that the print server expects 
PostScript (in fact, most users have no idea what PostScript is). In response, administrators change their 
print filters to automatically convert plain-text input files to PostScript so that users can use lpr to send 
either a PostScript or plain-text file to the printer without worrying about conversions. 


Other operating systems don't usually let you get very close to the printer; instead, they make you print 
through an application that produces the correct output. Unix has these applications, too (such as gimp 
and its gimp-print counterpart), but Unix also lets you get very close to the printer with lpr. 


Users tend to want to print pictures and various other kinds of data. If you send a JPEG file or other file to 
a printer with lpr, it may do the right thing if the administrator set up the print filter to recognize and filter 
the file. More often than not, though, the print system either rejects the file or prints endless pages of junk, 
because the system tried to convert the binary file into PostScript using the text filter. 


Therefore, it is extremely important that you know where the system converts your file into PostScript. 
This is not a problem if you primarily use graphical applications to print, because these programs normally 
generate PostScript and send the result to a command such as lpr. However, if you like to use the 
command line, you must be very careful about what you send to the printer. 


CUPS is fairly smart about document conversions because it attempts to determine the MIME type of any 
input data before sending it to the printer. Furthermore, the system verifies that it can convert the data to 
something that the printer understands. If CUPS doesn't know how to convert the data, the CUPS server 
rejects the document rather than passing the printer data that it won't understand. 


12.4.1Network Printing 


You should have an idea of how network printing clients send data to print servers: 
=» Berkeley LPD clients send data to TCP port 515 ona print server. 


=» CUPS uses the Internet Printing Protocol (IPP). IPP is a TCP protocol that looks like HTTP (Hypertext 
Transfer Protocol). The standard IPP port is 631, but CUPS also comes with a daemon that accepts 
the LPR protocol on port 515. 


=» Most network-ready PostScript laser printers allow you to connect to TCP port 9100 and talk directly to 
the print engine. Most network-aware print filters use this port to send data. 


Regardless of the protocol, you should be especially vigilant with respect to open network ports on print 
servers. Filter out or firewall any open ports. There have been serious vulnerabilities in print servers in the 
past, and you can expect that many more will crop up in the future, especially with a system as new as 


CUPS. This also holds true for network-aware printers. You do not want an intruder to waste the money 
that you spend on printer supplies. 


12.5CUPS 
With these preliminaries out of the way, let's look at CUPS. CUPS has several interfaces, including a Web- 
based status and administration tool. The Web interface is very good for normal maintenance and operation, 


but unfortunately, if your needs lie outside the abilities of the interface, you still have to know how the system 
works and where the various configuration files reside. 


You can install CUPS anywhere you like. Most distributions use a /usr installation prefix, so the clients are in 
/usr/bin, the system programs are in /usr/sbin, and so on. This book assumes that your directory prefix 
is/usr. Here are the directory locations and contents: 


/usr/sbin The cupsd server and administration programs 

/usr/bin User-level clients, including lpr and 1p substitutes 

/etc/cups Configuration files 

/usr/1ib/cups Filters, backends, network frontends, and Web interface scripts 
/var/log/cups Printer log files 

/var/spool/cups Spooled print jobs 

/usr/share/cups PPD file sources, fonts, and other program data 


/usr/share/doc/cups Online documentation 


12.5.1Configuring cupsd 


The central server and scheduler in CUPS is named cupsd, and the central configuration file is cupsd. conf. 
You normally do not need to do anything to get a working cupsd, but a stock configuration file isn't usually 
very secure (see Section 12.5.2). 


Examine the configuration file to get an overview of the things you can change. If you have ever configured 
the Apache Web server, you will immediately observe similarities between the httpd configuration files and 


the CUPS files. 


Yourcupsd.conf file contains a lot of information, but first look for the AccessLog path and change it if you 
want the log file in some other place: 


AccesshLog /var/log/cups/access_log 


The default IPP port is 631, so you should see the following line: 
Port ovl 


You should not need to change this. However, you may wish to add more ports with the Listen keyword. 


The following parameters select the user and group that run all tasks except the cupsd server. You likely do 
not need to change these, but you may need to add the user and group to your system: 


User lp 
Group Sys 


12.5.2CUPS Security 


Before you start CUPS, you should get a handle on security matters. Configured and installed from source 
code, CUPS does not have an enviable level of security, and you should do something about that before you 
even start the server. If your CUPS installation came with a Linux distribution, you should still take a look 
through this section to verify that you have a practical level of security. 


This section covers how to configure digest authentication, the most useful type of authentication available for 
the CUPS Web administration interface. Run through the following steps on a new installation: 


1. Scan your cupsd.conf for a section that looks like this: 


<Location /admin> 
AuthType Basic 
AuthClass System 


## Restrict access to local domain 
Order Deny,Allow 

Deny From All 

Allow From 127.0.0.1 

#Encryption Required 

</Location> 


2. ChangeAuthType BasictoAuthType Digest. Basic authentication sets up a Web browser to 
send Unix user passwords as plain text. As earlier in the networking chapters, this is a really bad idea. 


3. Run the following command to add an admin user to the CUPS digest password file 
(/etc/cups/passwd.md5): 


lppasswd -a admin 


4. When prompted, choose a password containing at least six characters and one number. 


Certificate Authentication (for Command-Line Utilities) 


You are now set up to use the CUPS Web interface securely. However, you will inevitably turn to command- 
line administration programs such aS lpadmin when managing CUPS. These utilities also require 
authentication because they operate through the same network port as the Web interface. However, you may 
notice that you don't need a password to run these command-line programs as the superuser. Why is this so? 


The answer is that CUPS has another form of authentication called certificate authentication, used primarily by 
the command-line programs and internal server communication. The CUPS scheduler places identifiers in the 
certificate directory /etc/cups/certs for various purposes, but the one you care most about is 
/etc/cups/certs/0. Any user that can read this file can run administrative programs without a password, 
because these programs send the certificate instead of a username and password. 


Note Don't use chmod to allow administrative access for non-root users because you may inadvertently 


allow administrative access for anyone on the system. Instead, add the desired users to the Unix 
group specified by the Group directive from Section 12.5. 


12.5.3Starting CUPS 


With your security situation now under control, you're ready to start the CUPS scheduler and add some 
printers. To start CUPS, all you need to do is run cupsd as the superuser. If you compile CUPS from source 
code, you should take careful note that the install procedure modifies your init.d and rc.d boot directories 
to make CUPS part of your startup sequence. 


To test your installation, access the Web interface by pointing a Web browser to CUPS at this URL (notice that 
this is the IPP port on your machine): 


http://localhost:631/ 


This should yield a screen with several printing options. There are no printers ina new CUPS installation, so 
adding a printer is your first order of business. 


12.5.4Adding and Editing Printers 


The easiest way to add a printer is with the Web interface (there is an AddPrinter button on the printer list 
page). The first time you try to do any administration task, CUPS asks you for a username and password. Use 
theadmin user and password that you configured in Section 12.5.2. 


To add a printer with the command-line interface, use the lpadmin command. The bare minimum command 
for adding a printer without a print filter is as follows: 


lpadmin -p name -v device 


Depending on your configuration, you may need the -E option to specify encryption. 


Adding a Test Printer 


As is the case with most Unix utilities, commands like 1padmin do nothing more than modify text files (except 
that with CUPS, 1padmin opens a network connection to the CUPS server, sends some data, and then the 
server changes the files). You should know exactly what happens when you run lpadmin. 


Add a dummy test printer by going through the following steps: 
1. Runlpadmin to add the printer: 


lpadmin -p test -v file:/dev/null 


2. Uselpstat to verify that the printer is there: 


Ipetarc =p 


Your new printer should show up as follows: 


printer test disabled since Jan 01 00:00 - 
reason unknown 


3. Examine/etc/cups/printers.conf. You should see an entry like this: 


<DefaultPrinter test> 
Info test 
DeviceURI file:/dev/null 


</Printer> 


4. Activate your printer: 


lpadmin -p test 


Thelpstat -p command should now report the printer as idle rather than disabled. 


5. Send a test job to the printer: 


echo test | lp -d test 


6. Verify that the job started by looking at /var/log/cups/error_log. The file-name is somewhat 
misleading, because error_log also contains normal diagnostic messages. You should see some 
entries that look like this: 


I [05/0ct/2003:14:44:46 -0700] Started "/usr/lib/cups/cgi- 
bin/printers.cgi”™ (pid=17156) 

I [05/0ct/2003:14:46:23 -0700] Adding start banner page "none" to job 1. 
I [05/0ct/2003:14:46:23 -0700] Adding end banner page "none" to job 1. 

I [05/0ct/2003:14:46:23 -07001 Job 1. queued on ‘test’ by “user. 


7. The job should also complete. Use the following command to make sure this is the case: 


lpstat -W completed -1 test 


The output should look like this: 


test=-1 user 1024 Sun Oct 5 14746223 2003 
queued for test 


8. Remove the test printer when you're finished by using this command: 


lpadmin -x test 


This lengthy procedure confirms that your CUPS server is running and that you can add and modify printers. 


Adding a Real Printer 


Now that you have some familiarity with the process, you are ready to add a real printer with a particular 
model using this command: 


lpadmin -p name -v device -m model 


Themodel parameter is extremely important because it provides printer-specific information, including the 
printer driver specification. If you do not use the -m parameter, CUPS uses a "raw" print model, sending 
unfiltered input directly to the printer. You almost certainly do not want this, because printers can generate 
copious amounts of garbage upon receiving bad input. 


If you want to make the new printer the default, run this command: 


lpoptions -d name 


For the most part, you provide a PostScript Printer Description (PPD) filename as model. You can get a list of 
PPD files with this commana: 


lpinfo =m 


However, in a stock CUPS installation, it's unlikely that the output list will contain an appropriate PPD entry for 
your printer, even though some of the entries may look tempting. Before discussing PPD files (see Section 
12.5.6 for more information on those), let's briefly look at how to specify a device for the -v option in the 
earlierlpadmin command. 


12.5.5Printer Devices (Backend Specifications) 


As you might surmise from the file:/dev/nul1 specification in the test printer you created, you specify 
devices with Uniform Resource Identifiers (URIs). You can get a partial list of URI options with the lpinfo 
command: 


lpinfo ~v 


The output looks something like this: 


network socket 

network http 

network ipp 

network lpd 

direct parallel:/dev/1p0 

direct scsi 

serial serial:/dev/ttyS0?baud=115200 
serial serial:/dev/ttyS1?baud=115200 


The right-hand column contains backend types and complete URIs. You might see the following backends: 


parallel Most direct connections are still through the parallel port. The port named LPT1 in DOS and 
Windows is /dev/1p0 on Linux, so the following device specification works fine in most cases: 


parallel:/dev/1p0 


However, you may want to use the bidirectional capabilities of the port for PostScript printers. For 
example, you may want an accounting program to ask the printer how many pages it prints for each job. 
Use this device for the bidirectional parallel port: 


parallel:/dev/parporto 


usb Newer printers with USB connections use a slightly different backend than parallel ports do. The URI 
looks something like the following, although the device filename may not be the same on your system: 


usb: /dev/usb/1p0 


socket This is a direct TCP network connection to the print engine of a network printer. Most of these 


printers listen on TCP port 9100, so you can specify this as follows (since port 9100 is the default, you 
don't really need the : 9100): 


socket://printer_host:9100 


Ipd If you want CUPS to send jobs to the Berkeley LPD port of a remote print server or printer, use this 
URI specification: 


lpd://host/printer_name 


smb This is for printing to a Windows printer share. You need to use Samba's smbspool program in this 
case. See Section 14.6.2 for more details on the URI specification and setting up the Samba backend. 


ipp, http These are for printing to other network IPP servers (such as remote CUPS servers). Here is an 
example: 


ipp://host/printer_name 


serial In the earlier days of Unix, machines talked to printers with a serial port, and some special-purpose 
printers still have serial ports. Because serial ports have a more complex system configuration than 
parallel ports, you often need special parameters to indicate the baud rate and other port settings. At the 
very least, you should specify the baud rate, as in this example: 


serial:/dev/ttyS1?baud=38400 


However, you often need to specify the parity and flow-control settings. Flow-control settings include soft 
(Orxonxoff),dtrdsr, and hard (or rtscts). Use a + sign before each additional setting, as shown in 
the following example: 


serial:/dev/ttyS1?baud=38400+size=8+parity=nonet+flow=dtrdsr 


scsi SCSI printers are rare, but if you happen to find one, you can print to its generic SCSI device, as in 
this example: 


scsi:/dev/sgl 


file This prints to a file. By default, CUPS disables printing to any file on the system except /dev/null. 
You can change this by adding a FileDevice parameter in your cupsd. conf file. Be warned that you 
risk very serious security breaches if you do so. 


CUPS does not let you choose a backend that it knows it cannot reach. For example, if your kernel does not 
have parallel port support, CUPS does not list parallel ports when you run lpinfo -v. However, the system 
isn't perfect; a port showing up in the list does not mean that support actually exists. 


Each one of these backend specifications uses its own separate backend program located in 
/usr/lib/cups/backend. The backend comes in at the very end of the print process. 


12.5.6PPD Files 


After choosing an appropriate device, you can complete the printer definition with a PPD file. CUPS uses PPD 
files for its printer database. 


A PPD file is in plain text and describes a printer's capabilities. CUPS keeps its default PPD files in 
/usr/share/cups/model; to make the options in a new PPD file available to the CUPS Web interface, put 
the file in /usr/share/cups/model and send your cupsd a HUP signal. When you import a PPD file into 
the CUPS PPD file database with lpadmin -m, CUPS makes a copy of the file in /etc/cups/ppd. 


The PPD file gives you fine control over printing options. Notice that if you choose the raw interface (that is, 
no PPD file), you get no options with the Configure Printer button in the CUPS Web interface, so you cannot 
change media, duplex, banner page, and other settings. 


Here is the general printer and file information from the PPD file of an Apple LaserW riter 16/600: 


*FormatVersion: "4.3" 
*FileVersion: eg 
*LanguageVersion: English 
*LanguageEncoding: ISOLatinl 


*PCFileName: "POOTSCR LPPD" 
*Manufacturer: "Apple" 
*Product: "(LaserWriter 16/600)" 


Looking elsewhere in the file, you might see some information on page sizes: 


*PageSize Letter/US Letter: "<</PageSize[612 792]/ImagingBBox null>>setpagedevice" 
*PageSize A4/A4: "<</PageSize[595 842]/ImagingBBox null>>setpagedevice" 


A standard PPD file should also contain font information: 


*Defaultront? Courier 

*Font AvantGarde-Book: Standard "(001.006S)" Standard ROM 

*Font AvantGarde-BookObligque: Standard "(001.006S)" Standard ROM 
*Font AvantGarde-Demi: Standard "(001.007S)" Standard ROM 

*Font AvantGarde-DemiObligque: Standard "(001.007S)" Standard ROM 
*Font Bookman-—Demi: Standard "(001.004S)" Standard ROM 

*Font Bookman-—Demiltalic: Standard "(001.004S)" Standard ROM 


*Font Bookman-Light: Standard "(001.004S)" Standard ROM 

*Font. Bookman-Light Italic: Standard "(001.0048)" Standard ROM 
*Font Courier: Standard "(002.004S)" Standard ROM 

*Font Courier-Bold: Standard "(002.004S)" Standard ROM 


The preceding excerpts are from a GPL version of a PPD file. PostScript printer manufacturers provide PPD 
files for their products, and for the most part, these should work fine with CUPS. However, non-PostScript 
printers do not normally come with PPD files, so they need a third-party PPD file that specifies a print filter to 
rasterize PostScript. You can get CUPS PPD files at sites such as http:/Awww.linuxprinting.org/. 


12.5.7Specifying a Filter 


One of the most frequently encountered problems with CUPS is that many of the PPD files you find on the 
Internet contain CUPS-specific print filter parameters that you may not have on your system. You should be 
able to recognize these parameters and have an idea of how to correct or delete them as you go. 


CUPS filters convert file formats in two ways: 


To a common format This type of filter converts various file formats into a common format. This 
common format will be either PostScript or raster data. The mime. convs file controls the filters that 
perform the conversion. For the most part, you should not have to worry about these filters, which are 
covered in the next section. 


From a common format A filter specified in the PPD file runs just before the backend to change the 
common format data into the language that the printer understands. If you have a PostScript printer, you 
probably do not need this filter in your PPD file. However, if you don't have a PostScript printer, these 
backend filters require a lot of attention, as you will see. 


CUPS Filters and mime.convs 


CUPS comes with a number of filters in /usr/lib/cups/filter that transform various intermediate file 
formats into printer-ready data. However, it is difficult to see where and how CUPS runs these filters. 


When you print a file, you can observe something like the following in the log file (this example is for a text 
file): 


Started filter /usr/lib/cups/filter/texttops (PID 3999) for job 25. 
Started filter /usr/lib/cups/filter/pstops (PID 4000) for job 25. 


To get this detailed logging, the LogLevel parameter in your cupsd. conf must be a minimum of info. 
However, info does not log the initial input file type; you need to set the parameter to debug in order to log 
this additional information. 


In the log, you can see some filters running, and from the names of the filters, you know that CUPS has some 
sort of idea about the job's file type. Unfortunately, the actual workings of the conversion mechanism aren't 
terribly obvious from the log file output. Here's how the sequence works: 


1. CUPS determines the output format that the backend needs to send to the printer by looking at the PPD 
file. 


2. CUPS determines the input file type with the help of /etc/cups/mime.types. In the preceding 
example log output, the application/postscript rule in /etc/cups/mime.types matched. 


3. CUPS looks through the /etc/cups/mime.convs file for a suitable filter to perform the conversion 
between the input file format and the format that the backend needs. Consider this line from 
mime.convs that transforms plain-text files to PostScript output: 


text/plain application/postscript 33 texttops 


The four columns here are the input MIME type, the output MIME type, the estimated cost, and the 
name of the filter program that converts the input to the output. 


4. CUPS runs the filter or filter sequence required for the printer backend. If there are no suitable entries 
inmime.convs, CUPS logs this message: 


Unable to convert file 0 to printable format for job n! 


5. CUPS runs any necessary additional filter before handing the final output to the backend (see the next 
section). 


Specifying a Backend Filter in the PPD File 


Just before sending the job to the printer backend, CUPS may run a backend filter from the PPD file. 


Backend filters are set with the cupsFilter parameter in the PPD file. Here is an example from the PPD file 
that you saw earlier: 


*cupsFilter: "application/vnd.cups-postscript 0. foomatic-rip" 


As you can see, the cupsFilter parameter's value consists of three parts: 
1. An expected MIME type. In the preceding example, the type is PostScript. The default is 


applications vnd.cups—postscript. 
2. The relative cost of the filter (set this to 0). 
3. The filter's executable name. 


The preceding cupsFilter parameter is very typical for printers that do not understand PostScript. Foomatic 
is a print filter that handles many different kinds of printers. 


There are a few gotchas with the cupsFilter setting: 


=» You don't need an extra filter if your printer already understands PostScript. Many PPD files for PostScript 
printers that you find on the Web contain filters that don't really do anything, but require extra software 
configuration. Delete any cupsFilter lines in your PPD file if your printer understands PostScript. 


=» The default PPD files and mime. convs file that come with CUPS do not contain the necessary settings to 
rasterize PostScript files, and it can be somewhat confusing to configure the rasterizer. For example, the 
generic HP LaserJet PPD files in the CUPS distributions expect raster (not PostScript) input. However, 
the default mime. convs file doesn't contain an active entry for transforming PostScript into raster output. 
For this reason, you may want to avoid the standard CUPS PPD files and look on the Web for appropriate 
files. 


= To address the problem of rasterization, most PPD files for non-PostScript printers on the Web assume 
that you have Foomatic as a filter (see Section 12.5.8) with Ghostscript as the rasterization engine. If your 
printer doesn't understand PostScript, it's unlikely that you will be able to avoid Foomatic. 


There is good news, though. Most PPD files that you find on the Web work fine if all of your software 
(Ghostscript, Foomatic, CUPS, and so on) is installed correctly. 


12.5.8Foomatic (for Non-PostScript Printers) 


If your printer does not have native PostScript support, there may still be one more piece of the puzzle to 
configure — the rasterizer filter. There are several options here, but the most popular system at the moment is 
Foomatic. 


Before going into details about Foomatic, you should be clear on the end goal here: you want a print filter to 
run a Ghostscript (gs) command to render PostScript input into bitmap form to send to the printer. The 
command arguments look something like this: 


gs =q -dSAFER -dNOPAUSE -dBATCH device_options -sOutputFile=- -= 


This seems simple enough, but unfortunately, device_options can be atremendous mess. There are 
countless combinations of printer data formats and options. Foomatic attempts to resolve these problems with 
a database containing options for nearly every printer model out there. You may see references to this 


database all over the Web but very little information on how to actually use the database. 


What most documentation fails to mention is that most administrators and users don't even interact with the 
database. All you need to worry about is building printer configuration data, such as PPD files. In fact, this 
work is already done for you when you visit http://www. linuxprinting.org/ to get a PPD file for your printer. All of 
the information that the Foomatic filter program (foomatic—rip) needs isin that PPD file. For example, here 
is a portion of the HP LaserJet 1100 PPD file that contains Ghostscript arguments: 


*FoomaticRIPCommandLine: "gs -q —-dBATCH -dPARANOIDSAFER -—-dQUIET —-dNOPA&& 
USE -SDEVICE=1ijs —-sljsServer=hpijJstAsBsC —-dijsUseOutputFD%Z —-sOutputFik&é& 
le=— _W 

*End 


Note SeeSection 12.6.3 for notes on certain HP printer drivers. 


Testing Foomatic 


To use Foomatic as a filter, you need the foomatic-rip program in lib/cups/filter (a symbolic link to 
another location works fine). You can verify that Foomatic produces the correct printer output by running 
foomatic-—rip onthe command line, generating printer output by hand. You can then send this data directly 
to the printer, bypassing the print spooler. 


First, locate your printer's PPD file (ppd_file) and find atest PostScript file to test with (test_file.ps). 
Run this command to create printer data intest.prn: 


foomatic—-rip —V ~pad pod file test. File.ps. > CESC prn 


The-v option specifies verbose operation so that you can see exactly what the filter is doing. If everything 
goes well, you should see the following at the end of the diagnostic messages: 


KID4 exited with status 0 
Renderer exit stat: 0 
Renderer process finished 


Closing foomatlic—rip. 


Successful completion also means that test .prn should now be a gigantic (probably binary) file that you can 
send directly to the printer device. 


12.5.9Administrative Command Overview 


With the mess of print filters hopefully behind you, it's time to learn some commands for day-to-day tasks. 
Compared to other Unix server systems, it isn't terribly important that you Know the syntax of every last 
command, because the Web administration interface does an adequate job in most cases (and is sometimes 
far preferable to the command line — for example, when doing printer media option configuration). 


However, you should develop at least a passing familiarity with the following commands (remember that each 
command has a manual page): 


lpadmin Adds, removes, and modifies printers. 
lpinfo Displays device and PPD file information (for use with lpadmin). 


lpoptions Sets printer options. To display a quick list of options for a particular printer, run this 
command: 


lpoptions =p printer -=l 


lpstat Displays printer and job information (after you create printers). To display a printer's job queue, 
use this command: 


lpstat -o printer 


To see the status of every printer on the system, use this: 


lpstat -a 
cancel Removes ajob. To remove all jobs from a queue, run this command: 


cancel -d printer -a 


disable Disables printing (but still possibly allows new jobs on the queue). 
enable Enables printing. 

reject Disables print queuing. 

accept Enables print queuing. 


lppasswd Modifies users and passwords (for digest authentication; see Section 12.5.2). 


12.5.10Client Access Control 


By default, CUPS allows printer and administration access only to localhost. If you have a network of 
machines that you want to grant access to your CUPS server, then you have to change your cupsd.conf file 


to reflect this. 


To get an idea of how per-client access control works, edit your cups. conf file and search for the line that 
reads<Location />. It should start with a section that looks like this: 


<Location /> 

Order Deny,Allow 
Deny From All 

Allow From 127.0.0.1 
</Location> 


TheLocation / section defines global access-control parameters for the CUPS server. In the preceding 
example, the directives work together to give access to localhost (127.0.0.1) and nothing else: 


Order Deny, Allow Allows you to specify hosts and networks that are denied and allowed access with 
theDeny and Allow directives. Further more, it also means that you can override Deny directives with 
Allow directives. This is the most useful mode in CUPS, and you should never change it. 


Deny From A11 Denies access to all clients by default. Again, you should not change this. 


Allow From 127.0.0.1 Makes an exception to the preceding Deny From A11 rule, granting access 
to localhost. 


If you want to allow access for all of your printers to more clients, you can just add another rule directive after 
theAllow From 127.0.0.1 line. The following rule directive grants access to the subnet defined by 
LO Os Ley 2556250625550. 


Allow From 10.04.12.0/2554.255.255.0 


You may have asmany Allow From client lines as you like, and you can specify the client parameter 
in several ways. The following five specification types are the most useful: 


=» Subnet notation, as inthe preceding 10.0.1.0/255.255.255.0 example 


=» IP addresses, such aS 10.0.1.3 


a» Fully qualified domain names, such as client.example.com 
=» Domain wildcards, such as *.example.com 
=» Network interfaces, such as @IF (eth0) 


Remember that the Location / sectionis for global access, including printer job control and other client 
tasks. Administrative tasks, on the other hand, have their own Location section (<Location /admin>) that 
specifically excludes connections from all hosts except localhost. In fact, you have already seen this section 
when you forced CUPS to digest authentication. Search for it in your cupsd. conf to verify that it allows 
administrative access to localhost, and to nothing else. 


Note You can specify per-printer client access by adding <Location/printers/name> access control 
sections, where name is the name of a printer. You'll probably need to experiment for a while to get it 
to work perfectly. 

12.5.11Auto-Discovery 
One of the more interesting CUPS features is automatic network printer discovery. This is called browsing. 


Most CUPS installations have browsing active by default, meaning that CUPS listens for messages from other 
print servers on the network. The cupsd.conf parameter is Browsing: 


Browsing On 


However, without additional help, your CUPS server will likely not make itself Known to the other print servers 
on the network. To make your CUPS server broadcast its presence, you need a BrowseAddress broadcast 
address parameter in cupsd.conf. For example, to send notification packets to as many hosts on the 
network as possible (essentially, as many as your routers allow), use this parameter: 


BrowseAddress 255.255.255.255 


You can think of a numeric broadcast address as a sort of inverse subnet combined with an IP address. For 
example, to send packets to the subnet defined by 10.1.2.0/255.255.255.0, try this: 


BrowseAddress 10.1.2.255 


This said, the best choice is likely the "local" interface selection that sends broadcast packets out to all 
network interfaces except PPP links: 


BrowseAddress @LOCAL 


If you have a server with several Ethernet interfaces, but do not want to broadcast on all of these interfaces, 
use @IF. For example, here's how to broadcast to eth1: 


BrowseAddress @IF(ethl) 


12.5.12Running an LPD-Compatible Server 


Browsing and broadcasting goes a long way toward allowing communication between various CUPS servers 
on a network, but if you want older clients running Berkeley LPD printing systems to be able to print to your 
CUPS server, you need to configure your server to allow jobs from these clients. 


To do this, run a special LPD server frontend called cups—1pd located in /usr/lib/cups/daemon. This is 
aninetd service (see Section 6.3), so you want a line like this in your /etc/inetd.conf (or the xinetd 
equiv alent): 


printer stream tcp nowait cups-user /usr/lib/cups/daemon/cups-lpd cups-lpd 


cups-—user is your CUPS User parameter from cupsd.conf (usually 1p). There is no built-in access control 
incups-—l1pd, but like any other network service, you want to make sure that your firewall rules have adequate 
coverage; see Section 5.13. Remember that printer is definedin /etc/services as port 515. 


12.5.13Troubleshooting CUPS 


When you have a problem with CUPS, it's particularly important to take things one step at a time. You may 
find this sequence helpful: 


1. Make sure that CUPS is working properly by creating a test printer with no filters (raw mode) and a 
device of file: /dev/nul1l1. Send atest file to the printer with the Web interface and 1p, and verify 
that the jobs complete with lpstat: 


lpstat -W completed -o test_printer_name 
lpstat -o test_printer_name 


2. Use your printer's PPD file to activate your filter. See Sections 12.5.4 and 12.5.6 for more detail if 
something goes wrong. Print more test files, again making sure that the jobs finish. 


3. Add the printer's actual device, and then try another test job. If necessary, try printing to a file, then 
sending the file directly to the printer. 


If a test print job fails and stalls, the printer state changes to "not ready," and you need to start the printer 
again to retry the job or run any new jobs. Do this with enable printer or the Web interface. Before you 
start changing files around, though, you should look at your log files, which are described in the next section. 


Error Messages 


The complex nature of the print spooler and filters makes it crucial that you pay careful attention to your log 
messages. The most important log file is /var/log/cups/error_log (the name is slightly misleading 
because the file also contains normal diagnostic messages). 


Looking at the log reveals this format: 


I [timestamp] Started filter /usr/lib/cups/filter/pstops (PID 3434) for job 26. 
D [timestamp] ProcessIPPRequest: 10 status_code=0 
E [timestamp] [Job 26] unable to open print file - : Permission denied 


The letter at the start of the line indicates the message severity or priority. The three letters in the preceding 
example are as follows: 


| An informational log message. These notices usually indicate some significant event, such as a filter 
invocation or job submission. 


D A verbose debugging message. You should not care about these unless you run into a problem. 


E An error message indicating that something doesn't work. The CUPS scheduler usually keeps running if 
there is an error. However, you may end up missing a feature or having a stalled printer job. 


You can change the log level with the LogLevel parameter in the cupsd. conf file. The default level is info 
in most installations. If you want the debugging messages, you must use debug. If you change your 
LogLevel parameter, remember to reload your configuration file after making any changes by sending a HUP 
signal to cupsd. 


Filter Problems 


The first thing you should do after running into trouble in the print filter stage is to make sure that the CUPS 
filters in /etc/cups/mime.convs function properly. 


To do this, first disable any filters that you might have in your printer's PPD file so that you can eliminate the 
backend filter from the list of things that can go wrong. You will probably only have such filters if you are using 
a non-PostScript printer. If that's the case, edit your printer's PPD file (the one in /etc/cups/ppd), looking 
fora cupsFilter parameter (see Section 12.5.7). Comment out the parameter by adding a >% just after the *; 
for example: 


*ScupsFilter: "application/vnd.cups-postscript 0 foomatic-rip" 


After making this change (and making cupsd re-read its configuration with a HUP signal), submit a test job to 
see if data can make it past the mime.convs filter stage to the printer device. Although you are not likely to 
run into any trouble with the CUPS-supplied filter programs in mime. convs, this process may help you 
uncover spool directory permissions problems (discussed in the next section) because these sorts of problems 
usually show up in the initial print filter stages. 


Having worked out any kinks not involving the cupsFilter filter, re-enable this parameter and look for more 
trouble: 


a Verify that the cupsFilter filteris inthe 1ib/cups/filter directory. 


=» For the Foomatic filter, use foomatic-rip -v manually from the command line to check for valid 
output (see Section 12.5.8). 


m Check that the filter has the correct path for any necessary program. 
a |f your filter relies on Ghostscript, ensure that you have the required driver by running gs —help. 
Spool Directory Permission Errors 


Look for this message in your error log: 


E [timestamp] [Job n] unable to open print file ~ : Permission denied 


CUPS normally runs its filters as the pseudo-user Ip and the group sys, so this user must have access to the 
spool directory /var/spool/cups in order to access intermediate data. When fixing the permissions, look for 
these problems: 

a Check all component directories of /var/spool/cups. Can the lp user access each of them? 

= Make sure that you have the Ip user in your /etc/passwd file. 


a Verify that lp belongs to the sys group. (Remember that you can change the pseudo-user and group with 
theUser and Group parameters of cupsd. conf.) 


Device Problems 


After running through the filter gauntlet, you are ready to send output to devices. Fortunately, there is 
relatively little that can go wrong at this final stage: 


=» Thelp user must have write access to any printer device directly connected to the system. In addition, 
check the kernel bootup messages and /proc to ensure that the kernel recognizes the device ports. 


=» For network printers, make sure that your host can reach the printer. If your device is the socket type on 
TCP port 9100, test the connection with this command: 


telnetprinter_address 9100 


Terminate a successful connection by pressing CONTROL-], then CONTROL-D. Unless you know the 
printer's native language, this connection does little good other than confirming that it can be made. 


12.6Ghostscript 


If you want to get a handle on printing under Unix, you should get to know Ghostscript. As mentioned before, 
most Unix programs that print generate PostScript output. Ghostscript is a PostScript interpreter that includes 
these features: 


= Bitmap rendering from PostScript code in several different output formats 
a PostScript file filtering 

=m Reading and writing to PDF format 

» Document previewing 


To get a feel for how Ghostscript works, run it with a file and no other arguments: 


gsfile.ps 


In addition to getting a preview window showing the first page of the file, you should also see this message bacl 
at your terminal window: 


>>showpage, press <return> to continue<< 


As the prompt suggests, pressing RETURN pages through the file. After reaching the final page, you get a GS> 
prompt; you may press CONT ROL-D or CONTROL-C to exit. 


There are several frontend programs, such as gv and ggv, that give a nicer interface to the Ghostscript previev 
Whether you use one of these frontend programs or not, running a preview can verify that you have a valid 
PostScript file to begin with. This can go a long way in tracking down problems, especially if your print spooler 
also uses Ghostscript to rasterize images for the printer. 


12.6.1Command-Line Options 


A full gs command line such as the one in Section 12.5.8 may seem a bit complicated, especially when it come 
to device specification. However, it's not hard to gain enough familiarity to make use of Ghostscript's filtering 
capabilities. 


Take a close look at the following command, which filters old.ps to create new. ps: 


gs =g —-dNOPAUSE -dBATCH -dSAFER -—-sSDEVICE=pswrite -sOutputFile=new.ps old.ps 


The first four options are typical for filters that run with no user input. Here's what they specify: 
-q Quiet mode; suppress any normal diagnostic output. 


—dNOPAUSE Do not wait for the user to press RETURN after each page. 
-dBATCH Terminate after interpreting all input. 

-dSAFER Disallow file alteration capabilities (to avoid security problems). 
The-dPARANOIDSAFER option disables file-read capabilities. 


The next two options usually go hand-in-hand when creating an output file: 


—sDEVICE=driver_name Use driver_name as the output driver. In this context, the term driver usually 


refers to a file format driver rather than a piece of code that manipulates devices, though there are 
exceptions (such as the x11 driver for the preview window). To see a list of drivers, rungs —help. 


Note Some drivers require other command-line parameters to operate correctly. Two notable example 
are the uniprint and ijs drivers. The uniprint driver comes with a number of . upp files 


(containing extra parameters) in the Ghostscript library directory. To use these settings, use 
@file.upp; for example: 


gs @rfrile.upp -g -GNOPAUSE -—dBATCH -sOutputFile=out 


-sOutputFile=file Write output to file (use - for standard output). If you ish to create one file per 
page of PostScript input, insert sd somewhere nto the name of file. Ghostscript substitutes the page 
number for %d. 


12.6.2More Ghostscript Tricks 


Here are a few more useful things that you can do with Ghostscript: 


= To transform a PostScript file into a PDF file, use the pdfwrite driver (Ghostscript comes with a ps2pdf 
script that does essentially the same thing): 


gs -q -dNOPAUSE -dBATCH -dSAFER -sDEVICE=pdfwrite -sOutputFile=file.pdffile.ps 


m Thepswrite driver (introduced in the previous section) is an effective way to make somewhat defective 
files print. If you can view the file with Ghostscript, the chances are good that a file filtered with the 
pswrite driver will print on even a finicky printer. 


=» You can combine several PostScript or PDF files into one by adding the additional filenames to the end of 
the command line. 


=» To change the output resolution (in dots per inch), use the -rapi or -rxdpixydpi parameters. The 
default resolution is usually 72 dpi. These commands generate larger bitmaps: 


gs -q -dNOPAUSE -dBATCH -dSAFER -r150 -sSDEVICE=pnmraw -sOutputFile=- file.ps 
gs -q -dNOPAUSE -dBATCH -dSAFER -r300x150 -sDEVICE=pnmraw -sOutputFile=- file.f 


12.6.3HP Inkjet Server 


If you have an inkjet or non-PostScript laser printer made by HP, then you almost certainly need the HPIJS (HF 
Inkjet Server) printer driver package. The main part of HPIJS is a program called hpijs. You can think of this 
as a plug-in driver for Ghostscript that translates Ghostscript's internal raster image format into printer formats. 
This driver is especially important if you are printing photographs or other graphics. 


You can tell if you need HPIJS by looking at the PPD file for your printer. If you see -sDEVICE=ijs in the 
Foomat icRIPCommandLine parameter, you need the driver. You can install it as a binary package or from 
source code; the only thing that matters is that Ghostscript can find the command in its path. 


Note Current versions of hpi js come with foomatic-rip anda set of PPD files. The best output quality 
likely comes from using these PPD files rather than random ones you see on the Internet. 


12./7Further Printing Topics 


This chapter is really only the tip of the iceberg when it comes to printing. Chapter 14 revisits printing to 
look at details of how to interface CUPS with Samba servers, but among the aspects not touched here are 
application configuration, complex color graphics support, and advanced PostScript topics, not to mention 
several other Unix print server packages. 


Here are some Web sites that can help you with your quest for Unix printer configuration: 
http ://www.linuxprinting.org/ This site has PPD files and documentation on many print servers. It 
also contains the Foomatic central information site. 
http ://www.cups.org/ This is the central CUPS Web site. 
http ://www.ghostscript.com/ A good source for Ghostscript software and topics. 


http ://gimp-print.sourceforge.net/ Gimp-print is a GIMP plug-in specifically designed for direct color 
printing. In certain cases, you may want to make gimp-print generate printer-ready data, bypassing 
Ghostscript rasterization completely with the 1p -oraw feature. (Gimp-print can also create 
PostScript output.) 


Chapter 13: Backups 


There is no single system for backups on Linux. There are many different kinds of archiving programs and 
formats. You already saw tar, one of the most popular archivers, in Chapter 1. The good news is that 
there isn't much more to making backups than what you already know; the bad news is that it's not too 
clear how you should get the archive to the physical medium. 


This chapter takes you through the basics of backups: what data you need to back up, what backup 
devices are available, what types of backups you can make, how to use the most common archivers, and 
how to use tape drives. 


13.1What Should You Back Up? 


Your backup priorities should be as follows: 
1. Home directories. Try to make daily backups of your home directories. 


2. System-specific configuration and data, such as the stuff in /etc and /var. This should include 
your machine's list of installed packages. 


3. Locally installed software (for example, /usr/local). You don't want to lose the work you put into 
custom software installations, so make a backup every now and then. 


4. System software from your distribution. This really is the least of your worries, because you can 
probably restore this part of the system by reinstalling the operating system. Furthermore, by the 
time your disk gets around to crashing so hard that you need to restore from backups, you probably 
want to install a newer version of the operating system anyway. 


13.2Backup Hardware 


In the formative years of Unix, there was only one reasonable kind of backup hardware: the tape drive. 
Tapes were inexpensive compared to fixed storage (hard drives), they were fairly reliable, and they could 
be taken off-site for additional data security. 


However, hard disk technology has outpaced tape drive technology in the past ten years. For PCs, tape 
drives are expensive compared to hard disks (and the tapes themselves are also expensive). In addition, 
tape drive capacities are far smaller than the disks now on the market. 


There are currently two popular backup alternatives to tape drives: 
= CD- and DVD-R/RW drives 
m An extra hard disk (internal or external) 


Both of these work, but you need to improvise, because there aren't any tried-and-true standard solutions 
for making backups to these devices. You still want to use a traditional backup utility (almost certainly tar) 


to create archives on the medium, but you need to find a way to manage the archives by yourself. 


That said, don't count out the tape drive just yet. A significant portion of this chapter deals with tape drives. 


13.3Full and Incremental Backups 


Before going into detail about backup hardware and archiving programs, you need to know about the two 
basic backup types: 


Full backup An archive of everything on a filesystem or directory 


Incremental backup An archive of all items that have changed since the last full backup 


In large-scale operations, there are more kinds of backups, and you may hear of cumulative and differential 
backups. These are levels between incremental and full backups — that is, an incremental backup may 
contain all of the changes since the last differential backup, and the differential backup, in turn, could 
contain all changes since the last full backup. However, for small systems where you do backups by hand, 
it is reasonable to deal only with full backups and the incremental backups that build on the full backups. 


The advantage to incremental backups is that they usually finish quickly and do not require much storage 
space. However, over time, the difference between the last full backup and the current state on the disk 
gets to be so significant that these advantages dwindle: the incremental backups start to take up as much 
space as the full backups, and it can be very difficult to merge an old full backup and a newer incremental 
backup if you ever need to restore a full set of files from the backups. In particular, files that you 
intentionally deleted between the two backups can reappear. Therefore, you need to do a full backups ona 
regular basis. 


Systems administrators generally keep a set of full and incremental backups so that they can restore files 
that may have only existed for a short period of time. For example, if you have daily incremental backups 
that go back 30 days and a full backup from at least 30 days ago, you can restore any file that was on the 
filesystem at backup time during that time period. Of course, if you don't Know the exact dates, finding the 
file can take a little time. 


To restore a complete set of files, you need to extract the latest full backup first, then the latest 
incremental backup (unless, of course, the full backup is by chance newer than the incremental backup). 


13.4Using tar for Backups and Restores 


InSection 1.18, you learned the basics of tar. As it turns out, you already know nearly all you need in order 
to do manual backups. If you want a comprehensive list of tar options, you can look at the (gigantic) GNU 
info documents. However, the good news is that you don't need to know very many options, and after reading 
this section, you should only need tar --help every now and then as a refresher. 


13.4.1Creating Archives 


With processors as fast as they are now, and disks getting ever larger, it doesn't make much sense to omit 
compression for any archive. Recall that you can create a compressed archive of a directory with one 
command: 


tar zcvf archive directory 


A large part of making a backup is figuring out where to put the archive. There are a few choices here: 
= For tape drives, see Section 13.6.2 for how to write the archive to the tape device. 
=» For backing up to a spare disk, you can write the archive as a file on the disk's filesystem. 


a |f you need to pipe the output to standard output (to feed it to another command, such as cdwrite), use 
- as the archive name. 
Here are some options to consider when running GNU tar: 
-—-excludefile Exclude file from the archive. 


—-exclude-fromfile Doesn't include in the archive any of the files listed in file. 


—-one-file-system Archives files from inside a single filesystem. This is handy for breaking up 
backups by partitions. 


-—-absolute-paths Doesn't strip the leading / from an absolute pathname. This has its uses, but only 
if you Know what you're doing. 


—-listed-incrementalfile Uses file as the incremental state file (see Section 13.4.2). 


One tricky part about adding these options to a traditional tar command is that you must insert a - in front of 
the regular one-character options. Therefore, if you want to create an archive of the misc directory, but 
excludemisc/scratch, you would have to run the following with -zcv£ rather than with zcvf, as you saw 
in the previous example. Here's how it would look: 


tar —-exclude misc/scratch -zcvf archive misc 


Note Remember that tar strips the leading / from an absolute pathname (unless you specify the -- 
absolute-paths parameter). Therefore, to exclude an item, you must match the path that goes 


into the archive, not necessarily the one on the system. Let's say that you wanted to create an 
archive of / without /usr. You must remove the / by yourself: 


tar —--exclude usr -zcvf archive / 


13.4.2Incremental Backups with tar 


To enable incremental backups with GNU tar, you need to create an incremental state file that contains 
information about each newly archived file. Here's how to do it: 
1. Run the following command to make a full backup in full_archive (state_file is the name of 
the state file; you must choose your own name): 


tar -—--listed-incremental state file -zcvf full_archive filel 


2. Make a copy of the original state file. For example: 


cpstate_file state_file_incr 


3. When it is time to make an incremental backup, run the same tar command as earlier, but with the 
state file copy that you created in step 2, state_file_incr: 


tar --listed-incremental state file incr -zcvf incremental archive filel 


This should produce a much smaller archive than the full backup, because tar compares the time(s) 
listed in state_file_incr (remember, it's a copy of the original state_file) against the 
modification times of the current files on the disk. 


4. Repeat steps 2 and 3 for subsequent incremental backups. 
5. To do another full backup, delete or rename state file and start at step 1. 


You need to make a copy of the original state file before each incremental backup because GNU tar 
updates the file specified with --listed-incremental after its run. If you were to perform backups with 
the same state file over and over again, the incremental backups would only record the changes since the 
last incremental backup, not since the last full backup. Although this would yield a complete set of changes, it 
would also make it nearly impossible to find a lost file inside a set of incremental backups. Furthermore, to do 
a complete restore, you would need to use a// of your incremental backups. 


Note Remember that incremental backups are only good when accompanied by the previous full backup. 
Make sure that you don't overwrite your full backup with an incremental backup. 


13.4.3Extracting Archives 


Recall from Section 1.18.1 that you can extract files from a compressed archive as follows: 


tar zxvf archive 


By default, tar extracts all files in the archive. However, you can limit the extraction to specific files and 
directories by adding their names to the end of the command line (you may needtorun tar ztvftogeta 
list of filenames): 


tar zxvf archive filel file2 


When extracting system files and home directories as root (for example, from a restore or when moving the 
things around), it is critical that you use the p option to preserve the original permissions: 


tar zxvpf archive 


Otherwise,t ar creates files using the current permissions mask (umask). This can be annoying at the very 
least, and could cause your system to malfunction at the worst. 


Warning Before extracting an archive, it's very important to check your current working directory against 
any files that the extraction process might create so that you don't accidentally clobber newer 
versions of files with older ones from the archive. When doing a partial restore, it's usually a 
good idea to change to an empty temporary directory first. 


13.5Backups to Non-Traditional Media 


Even though you have the ability to create your own filesystems on CD- or DVD-R/RW drives and spare 
hard disks, you should still stick with an archiver program like tar to make your backups. The primary 
reason is simple: tar preserves all file attributes, including permissions and ownership. In addition, it is 
easier to manage full and incremental backups with tar. 


If you choose to make backups with cdrecord (see Section 11.2.2, your best bet is probably to write a 
compressedt ar archive directly to the CD or DVD with a command like this: 


tar zcf - | cdrecord dev=dev cdrecord_args - 


You may need to experiment with the speed option to cdrecord, because the archiving command may 
not produce data fast enough for your recorder. 


Again, remember that this command does not create a filesystem on the disc. To read from the archive, 
you can just use the raw device. 


13.5.1Backups to Hard Disks 


Using a spare hard disk for backups is relatively easy. Create a regular filesystem on the disk and simply 
write your archives as regular files. You should be able to keep a fair amount of data on the spare disk if 
you use compression and incremental backups. 


13.6Tape Drive Devices 


Tape devices are character devices on a Linux system, going by a variety of filenames: 


a» SCSI tapes use the names /dev/st0,/dev/nst0,/dev/st1,/dev/nsti1, and so on. The SCSI 
tape drive interface and driver is widely regarded as the most reliable, but, of course, SCSI tape 
drives are also more expensive than others. 


» ATAPI tape devices start at /dev/ht0 and /dev/nht0. 
= There is limited support for tape drives on the floppy controller at /dev/f£t0 and /dev/ntf0. 


To identify a tape drive, look in your kernel messages for lines like this: 


Vendor: HP Model: C1533A Rev: 9503 

Type: Sequential-Access ANSI SCSi revision: 02 
St: Version 20010812, bufsize 32768; wrt 30720; Max init. buts 4, 5/g segs 16 
Attached scsi tape stO at scsi0, channel 0, id 4, lun O 


There are two versions of each tape device on a Unix system: 


=» Arewind tape device rewinds the tape after every operation. For example, /dev/st0O is a rewind 
device. 


=» Ano-rewind tape device does not rewind the tape after an operation. No-rewind devices start with n; 
for example, /dev/nst0 is a no-rewind device. 


For day-to-day operation, rewind devices are practically useless. You may as well forget that they exist; 
the next few sections show that you often need to reposition the tape. 


When working with tapes, you sometimes need to know the tape block size. Applications tend to write to 
the tape in a certain block size, and many tape drives do not allow you to read from the tape if you specify 
the incorrect block size. This is almost never a problem, because you normally write to and read from the 
tape with the same program. However, if you want to use dd to get direct access to tape files, you may 
need to specify the block size manually (see Section 13.6.7). 


Before you get started with a tape drive, you should set the TAPE environment variable to the no-rewind 
device name. This variable saves a lot of typing in the long run, in particular, with -f£ options for the mt 
andt ar commands (which are explained in the next section). 


13.6.1Working with Tape Drives 


A tape is one of the simplest forms of media available. You can think of a tape as a sequence of files with 
afile mark between each file, as shown in Figure 13-1. Notice how there is no file mark at the beginning of 
the tape and that the first file is numbered 0. 


Start of tape File marks End of tape ~———+ 


a Mi 


Figure 13-1: Files and file marks on a tape. 








A tape does not contain a filesystem. Furthermore, there is no information on a tape indicating the name of 
each file (though automated backup systems usually include a special tape index file at the beginning of 
the tape). When manually working with a tape, you work with file numbers and file marks, and nothing 
else. 


To access the data on a tape, you must manipulate the tape drive head.Figure 13-2 shows the initial 
position of the tape head. Let's say that your tape device is st 0, so you're using the tape device 
/dev/nst0. First, verify the tape position with the mt status command (the -f file option specifies 
the tape device file; remember that you do not need this if you set the TAPE environment variable): 


mt =f /dev/nst0 status 





4 
Tape head 


Figure 13-2: The tape head at the beginning of a tape. 


The output should look like this (the following output is from a DAT drive): 


SCSI 2 tape drive: 
File number=0, block number=0, partition=0. 
Tape block size 0 bytes. Density code 0x13 (DDS (61000 bpi)). 
Soft error count since last status=0 
General status bits on (41010000): 
BOT ONLINE IM_REP_EN 


There's a lot of information here: 
a [he current file number is 0. 
=» Thecurrent block number is 0, so the tape head is at the beginning of a file. 
=» You should ignore the partition; most tape drives don't support it anyway. 
=» The block size is 0, meaning that the tape drive does not have a fixed block size. 
a The density code indicates how much data can fit on the tape. 


=» The soft error count is the number of recoverable errors that have occurred since the last time you ran 
mt status. 


= General status bits explain more about the state of the tape and tape drive. BOT means "beginning of 
tape," and ONLINE reports that the tape drive is ready and loaded. See Section 13.6.6 for a full list of 
mt Status bits. 


13.6.2Creating Archives ona Tape 


It is difficult to understand how a tape drive works by running through various files on a random tape in a 
collection of backups. Therefore, to explain the process of searching through a tape, this section shows 
you how to create some archives for yourself before going into the specifics of unarchiving. Once you 
have learned this, you will be able to play with a tape without significant confusion or worrying about losing 
data. 


Here is a procedure for creating three files on a tape — specifically, tar archives of /1ib,/boot, and 
/dev: 


1. Put a new tape in your drive (a blank tape, or one that you don't care about). 

2. Make sure that your TAPE environment variable is set to the no-rewind tape device. 
3. Runmt status to verify that the tape isin the drive and your TAPE variable is set. 
4. Change to the root directory. 
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. Create a tar archive of the first file on the tape with this command: 


tar zev lib 


6. Runmt status. The output should look something like the following, indicating that the tape is on 
the next file (file 1): 


SCSI 2 tape drive: 

File number=1, block number=0, partition=0. 

Tape block size 0 bytes. Density code 0x13 (DDS (61000 bpi)). 
Soft error count since last status=0 


General status bits on (81010000): 
FOF ONLINE IM REP_EN 


7. Create an archive of your /boot directory: 


tar zev boot 


8. Create an archive of your /dev directory: 


tar zcv dev 


The contents of your tape should now look like Figure 13-3. 
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Figure 13-3: The tape containing three archive files with the head at the start of a new file. 


13.6.3Reading From Tape 


Now that you have some files on the tape, you can try reading the archive: 


1. Change tothe /tmp directory to avoid any accidents (in case you accidentally type x instead of t in 
one of the steps that follow). 


2. Rewind the tape: 


mt rewind 


3. Verify the first file on the tape, the archive of /1ib: 


tar ztv 


A long listing of the files in the archive should scroll by. 


4. Runmt status. The output should look like this: 


SCSI 2 tape drive: 

File number=0, block number=4557, partition=0. 

Tape block size 0 bytes. Density code 0x13 (DDS (61000 bpi)). 
Soft error count since last status=0 

General status bits on (1010000): 

ONLINE IM_REP_EN 


The tape is still at file 0, the first file on the tape. How did this happen? The answer is that as soon as it 
finishes reading, tar stops where it is on the tape, because you may want to do something crazy (Such as 
append to the archive). The tape head is now at the very end of file O (notice the block count in the mt 


output); see Figure 13-4. 
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Figure 13-4: The tape head at the end of file O. 


The most important consequence of the new tape position is that another tarztv does not access the 
next archive on the tape, because the head is not at the beginning of the next file, but rather, it is at the 


beginning of the file mark. To advance the tape forward to the next file (the /boot archive), use another 
mt command: 


mt fsf 


This command moves the tape so that the head is at the end of the file mark (the beginning of the /boot 
archive), as shown in Figure 13-5. 
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Figure 13-5: The tape head at the beginning of file 1. 





13.6.4Extracting Archives 


To extract archives from a tape, all you need to do is replace the t in the commands in the previous 
section with xp: 


tar zxpv 


As usual, you should double-check your current working directory before extracting any archives. 


13.6.5Moving Forward and Backing Up 


Themt fsf command moves the tape to the start of the next file. There is also a command for moving 
the tape back: 


mt bsf 


However, itis important to understand that this command rewinds the tape to the end of the previous file 
on the tape; that is, to the start of the next file mark that the tape head sees. 


Let's say that you are at the start of file 1, as shown earlier in Figure 13-5. Running mt bsf reverses the 
state to that shown in Figure 13-4. Try it for yourself, and then run another mt status. The file number 
should be 0 with a block number of -1 (the driver cannot compute the length of the file by itself). 


In general, to get back to the start of the previous file when you are at the start of a file, you need to move 
back twice and then forward once. Because most mt tape movement commands take an optional file 
count (an example is the 2 here), you can run the following sequence: 


mt bsf 2 
mt fsf 


However, there's another gotcha here, because the tape in this example is at the start of file 1. Running mt 
bsf 2 attempts to move the tape back two file marks. However, the beginning of the tape is not a file 
mark. Therefore, the mt bsf 2 in this sequence backs up, passing one file mark, and then hits the 
beginning of the tape, producing an I/O error. The subsequent mt fsf skips forward again, moving the 
tape back to the beginning of file 1. 


Therefore, if you want to get to file 0, you should always rewind the tape instead with mt rewind. 


Note The Linux mt includes a bs fm command that is supposed to take you directly to the start of the 


previous file. This nonstandard extension does not work quite correctly; if the tape head is at the 
start of afile, mt bsfmruns mt bsf,thenmt fsf, taking you right back to where you started. 


The bottom line is that you need to pay careful attention to the tape head position with respect to the file 
marks and choose the appropriate command. If it all seems a little ridiculous to you, it's probably not worth 
sweating over; to get to file n ona tape, this always works (though it may be slower): 


mt rewind 
mt fsf n 


To eject the tape, use the offline command: 


mt offline 


13.6.6mt Commands and Status 


These are the most common mt commands: 
mt status Produces a status report. 


mt fsf n Winds forward to the start of the first block of the next file. The n parameter is optional; if 
specified,mt skips ahead n files instead of just one. 


mt bsf n Winds back to the end of the previous file. The n is an optional file count. 
mt rewind Rewinds the tape. 


mt offline Rewinds the tape, then prepares for tape removal. In some tape drives, the tape drive 
ejects the tape; in others, the drive unlocks the tape so that you can manually remove it. 


mt retension Winds the tape all the way forward and then rewinds. This sometimes helps with 
tapes that you're having trouble reading. 


mt erase Erases the entire tape. This usually takes a long time. 


There are countless other mt commands documented in the mt(1) manual page, but they really aren't very 
useful unless you're trying to extract blocks from the center of a file or to set SCSI tape driver options. 


Runningmt status yields a number of status bit codes (such as the BOT code from Section 13.6.1). Here 
are the important codes: 


ONLINE The drive is ready with a loaded tape. 
DR_OPEN The drive is empty (possibly with the door open). 
BoT The current position is the beginning of the tape. 


EOF The current position is the beginning of a file (the end of a file mark). This is a somewhat 
misleading code, because you can confuse it with the end of file. 


EOT The current position is the end of the tape. 


EOD The current position is the end of recorded media. You can reach this by trying tomt fsf past 
the very last file marker. 


WR_PROT The current tape is read-only. 


13.6.7Direct File Access 


tar isn't the only command that can write archives to a tape. Sometimes you may not know what is on the 
tape, and even if you do know, you may want to copy a file from the tape to the local filesystem for further 
(and easier) examination. 


Your good old friend dd can do this, as this example for /dev/nst0 shows: 


dd if=/dev/nst0 of=output_file 


This doesn't always work — you may get an I/O error because you did not specify the correct block size. 
The program that writes the archive on the tape usually determines the block size; in the case of tar, the 
block size is 10KB (specified to dd as 10k). Therefore, because dd defaults to a block size of 512 bytes, 
you would use the following command to copy a tar archive from the tape to output_file: 


dd bs=10k if=/dev/nst0 of=output_file 


Block sizes for common programs are listed in Table 13-1 (see also Section 13.7). 


Table 13-1: Block Sizes for Common Archiving Programs 


Archiver | Block Size bsParameter 
tar 10KB (20 x 512 bytes) bs=10k 


dd 512 bytes lbs=512 
cpio 512 bytes bs=512 


dump 10KB (20 x 512 bytes) bs=10k 
Amanda 32KB bs=s2k 


Note Unliket ar,dd advances the tape head to the next file on the tape, instead of just moving it up 
to the file mark. 











13. 70ther Archivers 


Althought ar is the most common archiver on Linux systems, there are two other important systems 
called dump/restore and cpio that you should be aware of. The following sections provide an overview of 
the most important archiver operations, but they do not go into very much detail. 


13.7.1Dump and Restore 


The dump/restore system is a throwback to the older days of Unix. Unlike most other archivers, the dump 
program looks at the internal structure of a filesystem, and therefore it only supports certain filesystem 
types. On Linux, this list currently includes only the ext2 and ext3 filesystems, and this is unlikely to 
change, because there is little interest in porting the system to the myriad other filesystems. This 
inflexibility makes dump/restore a Somewhat unattractive backup solution. Still, some people prefer dump 
(probably because they're too old to learn new tricks, or perhaps that they don't like their backup program 
to change the access times on their files), so you may be stuck with reading one of these archives. 


Dump 


Thedump command to back up an entire directory looks like this: 


dump -0O -f archive directory 


You can use a tape device as archive if you like, or if your tape device is in the TAPE environment 
variable, you can omit -farchive entirely. 


The-0 option forces a full (level 0) backup. If you actually intend to use dump to do regular backups, add 
a-u option to make dump write date information to /usr/etc/dumpdates. With this file and option in 
place, you can specify backup levels other than O (for example, a level 5 with —5). 


Note When using -u, you must dump the entire filesystem — that is, the directory parameter must 
be the mount point or the disk device (for example, /dev/hda4). 


Restore 


Therestore program extracts files and directories from an archive created with dump. If you're only 
looking for one or a few files, you need to run restore in interactive mode, so follow this procedure: 


1. Run the following command, where archive Is the archive (this can be a tape drive): 


restore itavt archive 


2. Wait until restore has read the filesystem index from the archive and prints arestore > 
prompt. 


3. Use the cd and 1s commands until you find a file or directory that you want to restore from the 
archive. 


4. Add the file or directory to the extraction list with the add command (add as many files and 
directories as you like): 


addfile 
5. Typeextract to begin the extraction. 
6. After extraction, restore asks this question: 


set owner/mode for '.'? [yn] 


If you want to set the current directory's permissions and owner to the same as the root directory in 
the archive, answer y. This is nearly always a bad choice for an interactive restore, so answer n if 


you have any doubt. It's not hard to fix this if you do make a mistake, though. 


Note The root directory in the archive corresponds to the originating filesystem's mount point. Let's say 
that you dump a partition mounted at /home. None of the paths in the archive would contain 
/home— that Is, /home/user would show up as /user in the archive. 


To forgo interactive mode and restore everything (and set the permissions for the current directory), use 
thervf options: 


restore rvf archive 


13.7.2cpio 


Linux administrators rarely use cpio (or its newer, more capable cousin named afio) for backups or 
other tasks, but you should still have an idea of how this program works so that you can create, test, and 
extract archives. 


cpio has three modes: 


= Create, or "copy-out," mode (activated with -o) takes a file list from the standard input, creates an 
archive from a matching directory hierarchy, and sends the result to standard output or a file. 


= Extract, or "copy-in," mode (activated with -i) tests and extracts files from an archive. 


m Pass-through mode (activated with —p) copies files. tar and rsync (see Chapter 15) usually do this 
job better unless you have a very specific list of files. This book does not cover this somewhat 
obscurecpio mode. 


Creating Archives 

Because you are unlikely to have a cpio file lying around on your system, you should experiment by 
creating one yourself. The cpio copy-out mode requires that you generate a file list first. You can use the 
output of find to create a file list in a pipeline to cpio. 


Let's say that you want to archive /usr/local/bin asa file named local_bin.cpio: 


find /wusrt7lodal/jbin |. cpio. =o > 1ocal._bin.cpio 


On any halfway modern system, the output of this command includes lines like this for every single file 
created: 


cpio: /usr/local/bin/file: truncating inode number 


This annoying message isn't anything to worry about; it just means that cpio can't store the full inode 
number in the limited space available in the archive. The very end of the command should contain a block 
count (by default, 512 bytes equal one block), as in this example output: 


20967 blocks 


Note If you need to write an archive to a tape, you have two choices (these are equivalent because 
the block sizes in cpio and dd are the same): 


cpio =0 -O tape 
cpio -o | dd of=tape 


Testing and Extracting Archives 


Now that you have local_bin.cpio, you can list its contents with this command: 


Cole =i, =f =y = Local Dinwcoie 


The options have the following meanings: 
-i Specifies copy-in mode. 


-t Specifies a test archive. 
-v Sets verbose mode; shows the files as 1s -1 would. You may omit this option if you don't care. 
-Ifile Reads file asthe archive (the default file is the standard input). 


To extract an archive, replace the -t with —d: 


Colo =i. -d =v -I archive 


However, this can be dangerous with the local_bin.cpio archive that you created earlier, because that 
archive contains absolute pathnames. This can be a threat to system stability and security, especially if 
you extract as root, because the extraction can easily overwrite a file such as /etc/passwd. Use the -- 
no-absolute-filenames option to prevent this: 


cpio =—=—no-absolule—-filenames ~i -d -~y =I local bin.cpio 


Keep in mind that there are a few more cpio options that cpio may need in copy-in mode if it can't figure 
out some of the format options of the archive for itself: 
-b Switches the byte order. The default cpio format depends on the byte order of the architecture of 
the CPU that was used to create the archive, so if you have an archive from a completely different 
kind of machine, you may need this option. 


-Cn Uses n as the archive block size. 


-Hname Uses name as the archive format (valid formats are bin,crc,hpbin,hpodc,newc,odc, 
tar, and ustar). Note that only GNU cpio supports all of these options. 


13.7.3Amanda Tapes 


Amanda is an automatic backup system that writes a "hybrid" archive on a tape. That is, the first 32KB 
block in an Amanda tape contains information about the archive, and the rest of the blocks make up a 
regular archive in tar or dump format. 
If you come across an Amanda file, do the following to extract the files: 

1. Put the first block into a file named header: 


dd if=file of=header bs=32k count=1 


2. Runstrings header. The output should look something like this: 


AMANDA: FILE 20031117 duplex /etc lev 0 comp .gz program /bin/tar 
To restore, position tape at start of file and run: 
dd if=<tape> bs=32k skip=1 | /bin/gzip -dece | bin/tar -f... - 


3. Follow the instructions. Extract the contents from the rest of the file as follows (if you're using a tape 
drive as file, the tape head should already be correctly positioned, so don't use skip=1): 


dd if=file bs=32k skip=1 | unpack_command 


unpack_command should follow the output you got in step 2. For example, if you're dealing with a 
tar archive, unpack_command might look like this: 


garp =a |). tar xpi = 


13.8Further Backup Topics 


This chapter covers the basics of manual backups for small systems. However, once you get into larger 
operations, there is much more to explore: 


a» Automated backups. Amanda (http://www.amanda.orq/) is a popular, powerful, and free automatic 
network backup package (that is, after you figure out how it works). 


= Tape changers. The Linux mtx command can operate a tape changer and autoloader. 
=» Backups with Samba. 

=» Commercial backup systems such as BRU. 

= Database backup systems (Oracle and the like). 


A good, comprehensive, and detailed guide to backups (especially for large organizations) is Unix Backup 
and Recovery [Preston]. 


Chapter 14: Sharing Files with Samba 


Overview 


Your Linux machine probably doesn't live alone on your network. If you have a PC running Windows, it's 
only natural that you would want to permit access to your Linux system's files and printers from those 
Windows machines with the standard Windows network protocol, SMB (Server Message Block). Mac OS X 
also supports SMB filesharing. 


The standard filesharing software suite for Unix is called Samba. Not only does Samba allow your 
network's Windows computers to get to your Linux system, but it works the other way around — you can 
print and access files on Windows servers from your Linux machine with the included client software. 


To set up a Samba server, you need to perform these steps: 
1. Create an smb. conf file. 


2. Add filesharing share sections to smb. conf. 
3. Add printer share sections to smb.conf. 
4. Start the Samba daemons: nmbd and smbd. 
Ashare is a resource that you offer to a client, such as a directory or printer. 


The discussion of Samba in this chapter is concise, limited to getting Windows machines on a single 
subnet to see a stand-alone Linux machine through the Windows Network Neighborhood browser. There 
are countless ways to configure Samba, because there are many possibilities for access control and 
network topology. For the gory details on how to configure a large-scale server, you will find How Samba 
Works [Lendecke] a much more extensive guide, and there is also information at the Samba Web site, 


http://www. samba.org/. 


14.1Configuring the Server 


The central Samba configuration file is smb.conf, and it is usually located in Samba's 1ib directory (for 
example,/usr/local/1lib). However, some distributions put the configuration file in an etc directory, 
such as /etc/samba Or /usr/local/etc. The smb.conf file breaks down into several sections 
denoted with square brackets (such as [global] and [printers]). 


The [global] section in smb.conf contains general options that apply to the entire server and all 
shares. These options pertain primarily to network configuration and access control. Here is a sample 
[global] section that shows you how to set the server name, description, and workgroup: 


[global] 
# server name 
netbios name = name 


# server description 
server string = My server via Samba 


# workgroup 
workgroup = MYNETWORK 
These parameters work as follows: 


netbios name The server name. If you omit this parameter, Samba uses the Unix hostname. 
server string A short description of the server. The default is the Samba version number. 


workgroup The SMB workgroup name. If you're on a Windows NT domain, set this parameter to the 
name of your domain. 


14.1.1Server Access Control 


There are a few options that you can add to your smb.conf file to put limits on the machines and users 
that can access your Samba server. The following list includes many options that you can set in your 
[global] section, as well as in the other sections that control individual shares (described later in the 


chapter): 
interfaces Set this to have Samba listen on the given networks or interfaces. Here are two 


examples: 
inmcerraces = 10.272 .2.07 2554 255142595.0 
interfaces = eth0O 


bind interfaces only Set this to yes when using the interfaces parameter in order to limit 
access to machines that you can reach on those interfaces. 


valid users Set this to allow the given users access. For example: 


valid users = jruser, bill 


guest ok Set this parameter to true if you would like a share to be available to anonymous users 
on the network. 


guest only Set this parameter to true to allow anonymous access only. 


browseable Set this to make shares available in Samba's browseable share list. If you set this 
parameter to no for any or all shares, you will still be able to access shares on the Samba server, but 
they will not be browseable, and therefore you will need to know their exact names. 


14.1.2Passwords 


In general, you only want to allow access to your Samba server with password authentication. 
Unfortunately, the basic password system on Unix is different than that on Windows, so unless you specify 
clear-text network passwords or authenticate passwords with a Windows server, you must set up an 
alternative password file. This section shows you how. 


Assuming that your Samba configuration directory is /etc/samba, you can use these entries in your 
[global] section to define the Samba password file as /etc/samba/passwd_smb: 


# use a separate password file for Samba to enable encrypted passwords 


security = user 
encrypt passwords = yes 
smb passwd file = /etc/samba/passwd_smb 


With these lines in place, you can manipulate the passwd_smb password file with the smbpasswd 
program. 


Note If you have access to a Windows NT domain, you can set security= domain, to make 
Samba use the domain's usernames and eliminate the need for an SMB password file. However, 
in order for domain users to access the machine running Samba, each domain user must have a 
local account with the same username on the machine running Samba. 


Adding and Deleting Users 


The first thing you need to do in order to give a Windows user access to your Samba server is to add the 
user to the passwd_smb file with the smbpasswd -a command: 


smbpasswd -a username 


Theusername parameter to the smbpasswd command must be a valid username on your Linux system. 


Much like the regular system's passwd program would, smbpasswd asks you to enter the new user's 
password twice. If the Samba password file does not exist, smbpasswd Creates it for you, then confirms 
that it has created the new user. 


To remove a user, use the -x option to smbpasswd: 


smbpasswd -x username 


At times, you may not necessarily want to delete a user, but rather, temporarily deactivate the user. The - 
d option disables a user; and -e re-enables the user: 


smbpasswd -d username 
smbpasswd -e username 


Changing Passwords 


You can change a Samba password as the superuser by using smbpasswd with no options or keywords 
other than the username: 


smbpasswdusername 


However, if the Samba server is running, any user can change their own Samba password by entering 
smbpasswd by itself on the command line. 


14.2Starting the Server 


After you add a user and a password, you should probably test your server. To start the server, you need 
to run nmbd and smbd with the following arguments, where smb_config_file isthe full path of your 


smb.conf file: 


nmbd -=D -s smb_config_file 
smod =D =S, SmO conrig Tile 


Thenmbd daemon is a NetBIOS name server, and smbd does the actual work of handling share requests. 
The-D option specifies daemon mode. If you alter the smb.conf file while smbd is running, notify the 
daemon of the changes with a HUP signal. 


14.3Diagnostics and Log Files 


If something goes wrong upon starting one of the Samba servers, an error message appears on the 
command line. However, runtime diagnostic messages go to the log. nmbd and log. smbd log files. 


These files are usually in a var directory, but you can change the location to dir by specifying the -1 
dir option to smbd and nmbd. However, if you want to override the log directory location (to /var/log, 
for example), it's usually better to change the log file directory and log filename at once with the logfile 
parameter in smb.conf. You can force the log output from smbd to a file with this line inthe [global] 
section of your smb.conf: 


log file = /var/log/smb.log 


Unfortunately, this does not set the nmbd log directory; if you want the log output from both daemons to go 
to the same file, you must still use nmbd -1 /var/log/smb.1log. 


14.4Sharing Files 


To export a directory to SMB clients (that is, to share a directory with aclient), add a section like this to 
yoursmb.conf file, where label is what you would like to call the share, and path is the full directory 


path: 


[label | 
path = path 
comment = share description 
guest ok = no 
writable = yes 
printable = no 


These parameters are useful in directory shares: 
guest ok Allows guest access to the share. The public parameter is a synonym. 


writable A yes or true Setting here marks the share as read-write. Do not allow guest access toa 
read-write share. 


printable Specifies a printing share; this parameter must be set to no or false for a directory 
share. 


veto files Prevents the export of any files that match the given patterns. You must enclose each 
pattern inside forward slashes (so that it looks like /pattern/). Here's an example that bars object 
files, as well as any file or directory named bin: 


velo tiles = 7 *..6/bi0n7 


14.4.1Home Directories 


You can put a special section called [homes] in your smb. conf file if you want to export home directories 
to users. The section should look like this: 


[homes] 
comment = home directories 
browseable = no 
writable = yes 


By default, Samba reads the logged-in user's /etc/passwd entry to determine the user's home directory 
for [homes]. However, if you don't want Samba to follow this behavior (that is, you want to keep the 
Windows home directories in a different place than the regular Linux home directories), you can use the 
%S substitution in a path parameter. Here is an example that switches a user's [homes] directory to 


j/u/ user: 


path = /u/%s 


Samba substitutes the current username for the 3S. 


14.5Sharing Printers 


You can export all of your printers to Windows clients with a special [printers] sectionin your 
smb.conf file. Here's how the section looks when you're using CUPS: 


[printers] 
comment = Printers 
browseable = yes 


printing = CUPS 
path = cups 
printable = yes 
writable = no 


To use the printing = CUPS parameter, your Samba installation must be configured and linked against 
the CUPS library. 


As you learned in Chapter 12, there are several other printing systems in addition to CUPS. On Linux, you 
can set the printing parameter to BSD,LPRNG,PLP, or CUPS. Of these, CUPS is the easiest. 


Note Depending on your configuration, you may also want to allow guest access to your printers with 
theguest ok = yes option because you may not want to give a Samba password or account 


to everyone who needs to access the printers. 


14.5.1Sharing a Single Printer 


If you want to export a single printer, you can do so with a printer section that looks like this: 


[printer_name] 
comment = My printer 
printer name = printer_name 
printing = CUPS 
path = cups 
browseable = yes 
printable = yes 


You can specify a driver name for Windows clients with the printer driver parameter. For example, 
to tell a client that the printer is an HP LaserJet 4M, you can use the following in the printer section: 


printer driver = HP LaserJet 4M 


14.6Using the Samba Client 


Samba comes with a program named smbclient that can print to and access remote Windows shares. 
This program comes in handy when you are in an environment where you must interact with Windows 
servers that offer no Unix-friendly means of communication. 


To get started with smbclient, use the -L option to get alist of shares from a remote server named 
SERVER: 


smoclient -L -U username SERVER 


You do not need -Uusername if your Linux username is the same as your username on SERVER. 


After running this command, smbclient asks for a password. If you want to try to access a shareasa 
guest, you can press ENTER; otherwise, enter your password on SERVER. 


Upon success, you should get a share list like this: 


Sharename Type Comment 

Software Disk Software distribution 
Scratch Disk Scratch space 

IPCs IPC IPC Service 

ADMINS IPC IPC Service 

Printerl Printer Printer in room 231A 
Printer2 Printer Printer in basement 


Look at the Type field to make sense of each share. You should only pay attention to the Disk and 
Printer shares (the 1PC shares are for remote management). In this list, there are two disk shares and 
two printer shares. You can use the name in the Sharename column to access the share. 


14.6.1Accessing Files 


If you need only casual access to files in a disk share, use the following command (again, you can omit the 
—Uusername if your Linux username matches your username on the server): 


smbclient -U username '\\SERVER\sharename' 


Upon success, you will get a prompt like this: 


smb: \> 


In this file transfer mode, smbclient is similar to the Unix ftp; you can run these commands: 
get file Copies file from the remote server to the current local directory. 


put file Copies file from the local machine to the remote server. 

cddir Changes the directory on the remote server to dir. 

ledlocaldir Changes the current local directory to localdir. 

pwd Prints the current directory on the remote server, including the server and share names. 


! command Runs command on the local host. Two particularly handy commands are ! pwd and ! 1s to 
determine directory and file status on the local side. 


help Shows a full list of commands. 


Using the SMB Filesystem 


If you need frequent access to files on a Windows server, you can attach the share directly to your system 
with the smbmount command: 


smomount '\\SERVER\sharename\' -c ‘mount mountpoint' -U username -P password 


To use this command, you must have the smbfs kernel driver loaded as a module or directly compiled into 
the kernel. In addition, the rest of the Samba programs should be in your path. By default, a Samba 
installation built from source code does not include the smbmount command. You must specify --with- 
smbmount when running the Samba configure script to create these extra programs. 


14.6.2Printing to a Windows Share 


The easiest way to print to a Windows SMB share is through CUPS. Follow these steps to get a printer 
working: 
1. Put the CUPS SMB backend in place. Find the smbspool program in your Samba distribution, and 
then create a symbolic link named smb in the backend directory to smbspool, like this: 


ln -s smbspool_path/smbspoolcups_prefix/lib/cups/backend/smb 


2. Restart CUPS. 


3. Runlpinfo -v to confirm that the SMB spooler is in the available backends. You should see this 
in the output: 


network smb 


4. Add a new printer as outlined in Chapter 12. If you don't need a password to access the printer, use 
the following as the printer device path (notice that the backslashes are forward slashes in CUPS): 


smb://SERVER/sharename 


If you need a username and password, add them to the front of the server: 


smb://username:passwordQSERVER/sharename 


If your target printer is in a different Windows workgroup, you can specify the workgroup in the first 
part of the name: 


smb://username:passwordQWORKGROUP/SERVER/sharename 


Sending Files Directly to a Printer Share 


If you have a file that is suitable for a printer (such as a PostScript file that you want to send to a 
PostScript printer), you can send it directly to the printer share with this smbclient variant: 


smbclient '\\SERVER\sharename' -U username -P < file 


This command is particularly useful when testing and troubleshooting printers. Working with CUPS and 
Samba simultaneously can be difficult, so if you find yourself running into trouble when trying to access a 
printer on a Windows system, try going through these steps: 


1. Use the Windows system to print a test document to a file, then transfer file to your Linux system. 
2. Run the command in the preceding example to attempt to print the test file. 


3. If the file does not print, you have a problem on the Samba side. Otherwise, the problem lies with 
CUPS. 


Chapter 15: Network File Transfer 


Overview 


This book's primary focus is the single-machine environment. However, any modern machine is ona 
network. Eventually, you will want to transfer files from one Unix machine to another. There are many 
ways to move files around, but the choices usually come down to these: 


a» Using a simple command such as scp if you need to transfer just a few files. 


a Using an archiving program such as tar through an ssh pipeline to transfer a directory hierarchy. 
Here is an example: 


tar cBvf = directory | ssh remote_host tar xBvpf = 


This old method gets the job done, but it is not very flexible. In particular, after the transfer completes, 
the remote host may not have an exact copy of the directory. If directory already exists on the 


remote machine, and it contains some extraneous files, those files persist after the transfer. 


a Using a synchronizer system that duplicates an entire directory structure on the remote host. The two 
most popular synchronizer utilities are rsync and rdist. 


a Using old, clunky, insecure programs like ftp that you should not use. 

This chapter explains the rsync system. The main advantages of rsync over other systems are these: 
=m There is only one command to learn, rsync, which has a relatively simple syntax. 
a It offers relatively good performance. 


a It has multiple operating modes. 


15.1rsync Basics 


The base requirement of the rsync system is that you install the rsync program on the source and 
destination hosts. The easiest way to transfer files is to use a remote shell account. This section assumes 
that you want to transfer files to or from a machine to which you have SSH access. 


On the surface, the rsync command is not much different than scp or rcp. In fact, you can run rsync 
with the same arguments as those utilities. For example, to copy a group of files to your home directory on 
host, you can run this command: 


PSynCritel Tilef ss» MOST 


However, given your current system and rsync configuration, this command probably will not work, 
because the rsync program defaults to using rsh as the remote shell, and as you saw in Section 6.7, you 
shouldn't use something this insecure. To get around this, you can tell rsync to use ssh with the --rsh 
option: 


rsync -—-rsh=ssh titel Tile- ses AOSC destination dir 


Look out for this error message: 


rsync not found 
rsync: connection unexpectedly closed (0 bytes read so far) 
rsync error: error in rsync protocol data stream (code 12) at 10.c(165) 


This notice says that your remote shell can't find rsync on its system. If rsyncisn't in the remote path 
but is on the system, use --rsync-path=path to manually specify its location. 


If you don't want to type -—-rsh=ssh every time that you invoke an rsync command, set the RSYNC_RSH 
environment variable to ssh. The remainder of the commands in this chapter assume that you have done 
this. 


If your username is different on the remote host, add useré@ to the hostname, where user is your 
username on host: 


rsyncfilel file2 ... user@host:destination_dir 


With no extra options, rsync copies only files. In fact, if you specify just the options described so far and 
you supply a directory dir as an argument, you will see this message: 


skipping directory dir 


To transfer entire directory hierarchies, complete with the same symbolic links, permissions, modes, and 
devices, use the -a option, as in this example: 


Psyc ~d Gir NOst oes ct ifacion. dir 


If you're not too sure what will happen when you transfer the files, use the -n option to operate rsync 
without actually copying any files. The -n option implies the -v (verbose mode) option, showing details 
about the transfer and the files involved: 


rsync -na dir host:destination_dir 


The output looks like this: 


building file list ... done 
ml/nftrans/nftrans.html 

[more files] 

wrote 2183 bytes read 24 bytes 401.27 bytes/sec 


15.1.1Making Exact Copies of a Directory Structure 


By default, rsync copies files and directories without considering the previous contents of the destination 
directory. For example, if you transferred the directory d containing the files a and b to a machine that 
already had a file named d/c, the destination would contain d/a,d/b, and d/c after the rsync. To make 
an exact replica of the source directory, you must delete files in the destination directory that do not exist 
in the source directory, such as d/c in this example. 


Use the -—delete option to remove files from the destination that don't reside in the local directory, like 
this: 


rsync =a -—-delete dir host:destination_dir 


Note Again, if you're not certain about your transfer, use the -n option to tell you if rsync wants to 
delete any files before it actually performs the removal. 


You may also wish to use -—max-—delete=n to make sure that rsync deletes no more than n files in the 
target directory. This option essentially amounts to a last-ditch effort not to delete large numbers of files 
when there's some problem on the source side. However, it will still delete files until reaching n deleted 
files, and in fact, you may notice that your version of rsync might have a bug, where it deletes one more 
thann files. 


15.1.2Using the Trailing Slash 


You have to be particularly careful when specifying a directory as the source in an rsync command line. 
Consider this command: 


rsync -a dir host sdesc_diz 


Once this command completes, you will have a directory dir inside dest_dir on host. However, adding 
a slash (/) significantly changes the behavior: 


rsync =a dir/host:dest_dir 


Here,rsync copies everything insidedir to dest_diron host without actually copying the dir 
directory itself. The dir directory will not exist on host after the operation. Therefore, you can think of a 
transfer of dir/ as an operation similar to cpodir/*dest_dir onthe local filesystem. 


Let's consider an example. Say that you have a directory a containing the files a and b (d/a and d/b). 
You run the following command to transfer them to the c directory on host: 


rsync =a d/ Aost:c 


After the transfer completes, the directory c contains copies of a and b, but nota. If, however, you had 
omitted the trailing / on d,c would have gotten a copy of d, with a and b inside, so that as a result of the 
transfer you'd have files and directories named c/d/a and c/d/b onthe remote host. 


When transferring files and directories to a remote host, accidentally adding a / after a path would 
normally be nothing more than a nuisance; you could then go to the remote host, add the dir directory, 
and put all of the transferred items back in dir. Unfortunately, you must be careful to avoid disaster when 
combining the trailing / with the --delete option. If you use the -—-delete option to transfer dir to 
dest_dir without the slash, rsync ignores everything in dest_dir except dir. Using dir/, on the 
other hand, makes rsync compare the contents of dir against dest_dir and makes dest_dir look 


exactly like dir. Therefore, you can accidentally delete a whole bunch of files in dest_dir. 
Warning Be wary of your shell's automatic filename completion feature. GNU readline and many other 
completion libraries tack a trailing slash onto a completed directory name. 


15.1.3Excluding Files and Directories 


One very important feature of rsync is its ability to exclude files and directories from a transfer operation. 
Let's say that you would like to transfer a local directory called src to host, but you want to exclude 
anything named RCS. You can do it like this: 


rsync -a —--exclude=RCS src host: 


It's important to note that this command excludes all files and directories named RCS because --exclude 
takes a pattern, not an absolute filename. If you want to exclude only one specific item, specify an 
absolute path that starts with /, as in this example: 


rsync =a —-exclude=/src/RCS src host: 


Note The first / in /src/RCsS in this command is not the root directory of your system, but rather, the 
very base directory of the transfer. 


Here are afew more tips on exclude patterns: 
=» You may have as many --exclude parameters as you like. 


= If you use the same patterns time and again, place those patterns in a plain-text file (one pattern per 
line), and then use --exclude-from=file. 


=» To exclude directories named item but include files with this name, use a trailing slash: —- 
exclude=item/. 


=» The exclude pattern is based on a full file or directory name component and may contain simple 
wildcards. For example, t*s matches this, but does not match ethers. 


a If you exclude a directory or filename, but find that your pattern is too restrictive, use -—include to 
specifically include another file or directory. 


15.2Checksums and Verbose Transfers 


Thersync command uses file checksums (near-unique signatures) to determine whether any files on the 
transfer source are already on the destination. 


The first time you transfer an entire directory hierarchy to a remote host, rsync sees that none of the files 
already exist at the destination, and it transfers everything. Testing your transfer with rsync -n verifies 
this for you. 


After running rsync, try running it again, but this time as rsync -n or rsync-v. This time, no files 
should show up in the transfer list because an exact copy of the file set exists on both ends, so the file 
checksums are the same. 


However, when the files on the source side are not identical to the files on the destination side, rsync 
transfers the source files and overwrites any files that exist on the remote side. There are a few options 
that override this checksum behavior: 


—-ignore-existing Does not clobber files already on the target side. 


--backup (abbreviation: -b) Does not clobber files already on the target, but rather renames 
these existing files by adding a ~ suffix to their names before transferring the new files. 


--suffix=s Changes the suffix used with -—backup from ~ to s. 


--update (abbreviation: -u) Does not clobber any file on the target that has a later date than 
the corresponding file on the source. 


With no special options, rsync operates quietly, only producing output when there is a problem. However, 
you can use rsync -v for verbose mode, and if you need even more output, rsync -vv gives yet more 
details. (You can tack on as many v options as you like, but two is probably more than you need.) Fora 
comprehensive summary after the transfer, use rsync --stats. 


15.3Compression 


Many administrators always use -z in conjunction with -a to compress the data before transmission: 


PSync ~az Git höostidestination dir 


Compression can improve performance in certain situations. For example, if you are uploading a large 
amount of data across a slow connection (such as the slow upstream link on many DSL connections), or if 
the latency between the two hosts is high, it helps. However, across a fast 100 Mb/s local area network, 
the two endpoints can spend more time compressing and decompressing data than the network takes 
transmitting the uncompressed files. 


15.4Limiting Bandwidth 


It's easy to clog the uplink of DSL and cable modem connections when uploading a large amount of data 
to a remote host. Even though you won't be using your (normally large) downlink capacity during sucha 
transfer, your connection will still seem quite slow if you let rsync go as fast as it can, because outgoing 
TCP packets such as HTTP requests will have to compete with your transfers for bandwidth on your uplink. 


To get around this, you can use —-bwlimit to give your uplink a little breathing room. For example, to 
limit the bandwidth to 10000 kilobytes per second, you might do something like this: 


rsync -—-bwlimit=10000 =a. dir host:destination_dir 


15.5Transferring Files to Your Computer 


Thersync command isn't just for copying files from your local machine to a remote host. You can also 
transfer files from a remote machine to your local host by placing the remote host and remote source path 
as the first argument on the command line. Therefore, to transfer src_diron host to dest_dir onthe 


local host, run this command: 


rsync =a NOSttsrce_dir dest dir 


Note If you omit host:, you can use rsync for making duplicates of directories on your local 
machine. 


15.6Further rsync Topics 


rsync Can do many more things that are beyond the scope of this book. One of its most significant 
abilities is acting as a network server. Such a server offers several modules that you can call by symbolic 
names instead of pathnames. You can also offer public read-only access to the modules; this is a nice 
alternative to anonymous FTP. 


Another important feature is rsync batch mode operation. Although it is fairly easy to write scripts 
containingrsync commands to do network file distribution, rsync can also employ a number of auxiliary 
files related to command options, logging, and transfer state. In particular, the state files make long 
transfers faster, and easier to resume when interrupted. 


There are also many more command-line options than those described in this chapter. To get a rough 
overview, run rsync --help. There is more detailed information in the rsync(1) manual page as well as 


at the rsync home page: http://samba.anu. edu. au/rsync/. 


Chapter 16: User Environments 


Overview 


This book's focus is on the Linux system that normally lies underneath a user's interactive session. 
Eventually, the system and the user have to meet Somewhere. You saw plenty of commands and shell 
features in Chapter 1 and Chapter 7, but there's still one little piece missing: the startup files that set 
defaults for the shell and other programs. 


Most users do not pay close attention to their startup files. Over time, the files become cluttered, 
containing unnecessary environment variables and tests that can lead to annoying (or quite serious) 
problems. Worse still, operating system vendors usually do not consider the consequences of what they 
put in the default startup files, packing them full of items that may even confuse the user. 


Keeping in tune with the shell-oriented theme of this book, this chapter focuses specifically on shell startup 
files. If you have ever received a Unix account in the past, it may have come with a bafflingly large array 
of startup files, sometimes called dot-files because they nearly always start with a dot (.). You may be 
surprised to learn that it really isn't necessary to have so many startup files. 


16.1Appropriate Startup Files 


When designing startup files, you should keep the user in mind. If you are the only user on a machine, you 
don't have much to worry about — if you make an error, you're the only one affected, and it's easy enough 
to fix. However, if your new startup files are to be the defaults for all new users on a machine or network, 
or if you think that someone might copy your files for use on a different machine, your task becomes 
considerably more difficult. If you make an error in a startup file for ten users, you might end up fixing this 
error ten times. That's annoying, to say the least. 


There are two essential goals to keep in mind when creating startup files for other users: 


Simplicity Keep the number of startup files small, and keep the files as small and simple as possible 
so that they are easy to modify but hard to break. Each item in a startup file is just one more thing that 
can break. 


Readability Use many comments in the files, so that the users get a good picture of what each part of 
a file does. 


16.2Shell Startup File Elements 


What goes into a shell startup file? Some things may seem obvious, such as the path and a prompt 
setting. But wait, what exactly should be in the path, and what does a reasonable prompt look like? And 
how much is too much to put in a startup file? 


The next few sections outline the essentials of a shell startup file, covering the command path, manual 
page path, prompt, aliases, and permissions mask. 


16.2.1The Command Path 


The most important part of any shell startup file is the command path. The path should cover the 
directories that contain every application of any general interest to a regular user. At the very least, the 
path should contain these components, in this order: 


Just) Local/bpin 
(ist / ban 
/usr/X11R6/bin 
(pin 


This order ensures that you can override standard default programs with site-specific program variants 
located in /usr/local. 


It's important to make every general-use program on the system available through one of the directories 
listed above (for example, using the Encap system described in Section 9.4). If the general-use programs 
are located in more than just these four directories, it probably means that your system is getting out of 
control. Don't change the path in your user environment to accommodate a new system directory. In fact, 
in the interest of minimizing the number of directories in the command path, some administrators prefer to 
fold the programs in /usr/X11R6/bin into /usr/local/bin or /usr/bin with the help of symbolic 
links or a different installation prefix. 


Many users use a bin directory to store their own shell scripts and programs, so you may want to add this 
to the front of the path: 


SHOME /bin 


If you are interested in systems utilities (such as traceroute,ping, and 1smod), add the sbin 
directories to your path: 


lasr/local/sbin 
/usr/sbin 
/sbin 


Adding Dot (.) to the Path 


There is one small but controversial command path component to discuss: the dot. Placing a dot (.) in your 
path allows you to run programs in the current directory without using . / in front of the program name. 
This may seem convenient when writing scripts or compiling programs, but it is a bad idea for two reasons: 


a It can be a security problem. You should never put a dot at the front of the path. For example, an 
attacker could put a Trojan horse named 1s in an archive distributed on the Internet. Even if a dot is 
at the end of the path, you are still vulnerable to typos such as s1. 


a It is inconsistent and can be confusing. A dot in the path can mean that a command's behavior can 
change according to the current directory. 


16.2.2The Manual Page Path 


Your manual path (the MANPATH environment variable used by man) should match your command path, 
except that each directory in the manual page path variable should end ina /man directory rather than 


Ibin: 


Here is the manual page path that corresponds to the command path listed in Section 16.2.1: 


/usr/local/man:/usr/man: /usr/X11R6/man: SHOME/man 


16.2.3The Prompt 


Avoid along, complicated, useless prompt. For whatever reason, many administrators feel the need to 
drag in everything, including the kitchen sink. Just because you can place the current working directory, 
hostname, username, and fancy decorations in a prompt, it doesn't mean that you should. 


Above all, avoid strange characters, such as these: 


Note Take special care to avoid >. This can cause erratic, empty files to appear in your current 
directory if you accidentally copy and paste a section of your shell window, because > redirects 
output from a file. 


Even a shell's default prompt isn't ideal. For example, the default bashprompt contains the shell name 
and version number, and the default tcsh prompt includes the hostname, the current working directory, 
anda >. 


Here is a simple prompt setting for bash that ends with the customary $(the traditional csh prompt ends 
with): 


Pole" aS 7 


The\u is a substitution for the current username (see the PROMPTING section of the bash manual page). 
Other popular substitutions include the following: 
\h The hostname (the short form, without domain names) 


\! The history number 


\w The current directory (this can become somewhat long; you can display only the final component 
with \ Ww) 


\$ Sif running as a user account, and # if root 


16.2.4Aliases 


Among the stickier points of modern user environments is the role of aliases, a shell feature that 
substitutes one string for another before executing a command. At one time, these were efficient shortcuts 
intended to save some typing. However, aliases also have these drawbacks: 


a They have balky syntax, especially when manipulating arguments. 


a They are confusing; a shell's built-in which command can tell you if something is an alias, but it won't 
tell you where it was defined. 


a They are frowned upon in subshells and non-interactive shells; they do not work in other shells. 


With all of these disadvantages, it makes sense to avoid aliases whenever possible, because it's easier to 
write a shell script (see Chapter 7). Modern computers can start and execute shells so quickly that the 
difference between an alias and an entirely new command should not mean anything to you. 


However, there is one particular instance where aliases do come in handy — when you wish to alter a part 
of the shell's environment. You can't change an environment variable with a shell script, because scripts 
run as subshells. 


16.2.5The Permissions Mask 


As described in Section 1.17.1, ashell's built-in umask (permissions mask) facility sets your default 
permissions. You should run umask in one of your startup files to make certain that any program you run 
creates files with your desired permissions. However, you may wonder what the best choice is, especially if 
you have a stand-alone machine with no other users. 


The two reasonable choices break down as follows: 


077 This is the most restrictive permissions mask; it does not give any other users access to new files 
and directories. This is appropriate on a multi-user system where you do not want other users to look 
at any of your files. 


022 Gives other users read access to new files and directories. This is more useful than you may 
initially presume on a single-user system, because many daemons that run as pseudo-users would not 
be able to see files and directories created with the more restrictive 077 umask. 


Note Certain applications (mail programs in particular) override the umask, changing it to 077 because 
they feel that their files are the business of no one but the file owner. 


16.3Startup File Order and Examples 


Now that you know what to put into shell startup files, it's time to see some specific examples. Surprisingly, 
one of the most difficult and confusing parts of creating startup files is determining which of several startup 
files to use. The next sections cover the two most popular Unix shells, bash and tcsh. 


16.3.1The bash Shell 


Inbash, you have these startup filenames to choose from: .bash_profile,.profile,.bash_login, 
and.bashrc. Which one of these is appropriate to use for your command path, manual page path, 
prompt, aliases, and permissions mask? The answer is that you should have a .bashrc file accompanied 
by a .bash_profile symbolic link pointing to .bashrc. 


The reason for this setup is that there are several different kinds of bash shell instance types, as follows: 


Interactive shell The is the type of shell that you use to run commands from a terminal. There are 
two subtypes of interactive shells: 


o Login shell This is traditionally invoked by /bin/login; for example, when getty or sshd 
Starts/bin/login. The exact circumstances that determine an interactive shell are a little 
strange (the first character of a login shell's invocation name is a -), but the basic idea is that the 
login shell is an initial shell. When bash runs as a login shell, it looks for a user's 
.bash_profile,.bash_login, and .profile files, running the first one that it sees. 


o Non-login shell This is any other interactive shell. Windowing system terminal programs (xterm, 
GNOME Terminal, and so on) start non-login shells unless you specifically ask for a login shell. 
bash reads from .bashrc upon startup of a non-login shell. 


Non-interactive shell This is a shell that doesn't require input from aterminal. 


The reasoning behind the two different startup file systems is that in the old days, users logged in through 
a traditional terminal with a login shell, then started non-login subshells with windowing systems or the 
screen program. It was deemed a waste to repeatedly set the user environment and run all sorts of 
wasteful programs in these subshells. With login shells, you could run fancy startup commands in a file 
such as .bash_profile, leaving only aliases and other "lightweight" things to your .bashrc. 


That's a nice theory, but in these modern times, nearly everyone logs in through a graphical display 
manager that never starts a login shell. Therefore, you need to set up your entire environment (path, 
manual path, and so on) in your .bashrc, or you would never see any of your environment in your 
terminal window shells. However, you a/so need a .bash_profile if you ever want to log in on the 
console or remotely, because these are login shells, and they don't ever bother with .bashrc. 


None of the issues described in the previous two paragraphs should matter, because it's never a good idea 
to make a complicated mess of your startup files, and modern hardware is so fast that a little extra work 
for every new shell shouldn't cause much of a performance hit anyway. 


Here is avery elementary (yet perfectly sufficient) . bashrc that you can also share with your 
¿bash profile: 


# Command path. 
PATH=/usr/local/bin:/usr/bin: /bin: /usr/Xl11R6/bin: SHOME/bin 


# Manual page path. 
MANPATH=/usr/local/man: /usr/man:/usr/X11R6/man:SHOME/man 


# PS1 is the regular prompt. 
# Substitutions include: 


# \u username \h hostname \w. current directory 
# \! history number \s shell name \$ $ if regular user 
PS1="\u\s ' 


# EDITOR and VISUAL determine the editor that programs such as less 
# and mail clients invoke when asked to edit a file. 

EDITOR=v1 

VISUAL=vi 


# PAGER is the default text file viewer for programs such as man. 
PAGER=less 


# These are some handy options for less. 
LESS=mei 


# You must export environment variables. 
export MANPATH EDITOR VISUAL PAGER LESS 


# By default, give other users read-only access to most new files. 


umask 022 


As described earlier, you can share this .bashrc file with .bash_profile via asymbolic link. One 
(possibly better) alternative is to create .bash_profile as this one-liner: 


SHOME/ -bashrc 


Checking for Login and Interactive Shells 


Now that your .bashrc matches your .bash_profile, you can no longer run extra commands for login 
shells. Therefore, if you want to define different actions for login and non-login shells, you can add the 
following test to your .bashrc, which checks the first character of the shell's $0 variable for a - character: 


case $0 in 
=K) command 
command 


esac 


If, for whatever reason, you want to run certain commands only if a shell is interactive, use a construct like 
this in your .bashrc: 


case "S-—" in 
KLF) command 
command 
rr 
esac 


Normally,bash doesn't read startup files at all for non-interactive shells. The preceding code only applies 
if you set the BASH_ENV environment variable to a startup filename. 


16.3.2The tcsh Shell 


The standard csh on virtually all Linux systems is tcsh, an enhanced C shell that popularized features 
such as command-line editing and multimode filename and command completion. Even if you don't use 
tcsh as the default new user shell (this book suggests using bash), you should still provide tcsh startup 
files in case your users happen to come across tcsh. 


You don't have to worry about the difference between login shells and non-login shells in tcsh. Upon 
Startup,t csh looks fora .tcshrc file. Failing this, it looks for the csh shell's .cshrc Startup file. The 
reason for this order is that you can use the .tcshrc file for tcsh extensions that don't work in csh. You 
should probably stick to using the traditional .cshrc instead of .tcshrc; it's highly unlikely that anyone 
will ever use your startup files with csh. And if a user actually does come across csh on some other 
system, your .cshrc will work adequately. 


Here is sample .cshrc file: 


# Command path. 
setenv PATH /usr/local/bin:/usr/bin: /bin: /usr/X11R6/bin: SHOME/bin 


# Manual page path. 
setenv MANPATH /usr/local/man:/usr/man:/usr/X11R6/man: SHOME/man 


# EDITOR and VISUAL determine the editor that programs such as less 
# and mail clients invoke when asked to edit a file. 

setenv EDITOR vi 

setenv VISUAL vi 


# PAGER is the default text file viewer for programs such as man. 
setenv PAGER less 


# These are some handy options for less. 
setenv LESS mei 


# By default, give other users read-only access to most new files. 
umask 022 


# Customize the prompt. 

# Substitutions include: 

# Sn username sm hostname %/ current directory 
# Sh history number sL- current terminal $5 5% 

set prompt="sm%ss " 


16.4Default User Settings 


The best way to write startup files and choose defaults for new users is to experiment with a new test user 
on the system. Create the test user with an empty home directory. Then, most importantly, refrain from 
copying your own startup files to the test user's directory. Write the new startup files from scratch. 


When you think you have a working setup, log in as the new test user in all possible ways (on the console, 
remotely, and so on). Make sure that you test as many things as possible, including windowing system 
operation and manual pages. After you're happy with the test user, create a second test user, copying the 
startup files from the first test user. If everything still works fine, you now have a new set of startup files 
that you can distribute to new users. 


The following sections outline reasonable defaults for new users. 


16.4.1Shell 


The default shell for any new user on a Linux system should be bash. In the old days, the Bourne shell 
(sh) was more difficult to use in interactive mode than alternatives such as the C shell (csh), so users 
typically had a login shell that was different than the standard system shell. This changed when bash 
arrived on the scene, for it had all of the features of csh and its enhanced version (tcsh). 


There are several good reasons for using bash ona Linux system: 


m Users interact with the same shell that they use to write shell scripts (csh is a notoriously bad scripting 
tool; please, don't even think about it). 


=» bash Is standard on Linux systems; tcsh is sometimes not. 


=» bash uses GNU readline, and therefore, its interface is identical to many other tools. tcsh has a 
powerful command-line editing system, but it takes some time to learn. 


=» bash gives you finer control over I/O redirection and file handles. 


However, you can't teach an old dog new tricks, and many seasoned Unix wizards use csh and tcsh 
simply because they can't bear to switch. Of course, you can choose any shell you like, but my opinion is 
that you should choose bash if you don't have any preference, and you should also use bash as the 
default shell for any new user on the system (if they have another preference, they will be able to change 
their own shell with the chsh command). 


Note There are plenty of other shells out there (rc,ksh,zsh,es, and so on). But just because they 
exist does not mean they are suitable beginner shells. 


16.4.2Editor 


The default editor should be vi or emacs. These are the only editors virtually guaranteed to exist on 
nearly any Unix system, and they will therefore cause the least trouble in the long run for a new user. 


As with shell startup files, avoid large default editor startup files. A little set showmatch in the vi .exrc 
startup file never hurt anyone, but steer clear of anything that significantly changes the editor's behavior or 
appearance, such as the showmode feature, autoindentation, and wrap margins. 


16.4.3Pager 


It is perfectly reasonable to set the default PAGER environment variable to less, because it is very easy to 
use. 


16.5Startup File Pitfalls 


There are a few things you really should avoid in startup files (many have to do with the X Window 
System): 


a» Don't put X commands ina shell startup file. 

=» Don't set the DISPLAY environment variable in a shell startup file. 

=» Don't set the terminal type in a shell startup file. 

= Don't skimp on descriptive comments in default startup files. 

a» Don't run commands in a startup file that print to the standard output. 


a» Never set LD_LIBRARY_PATH in a shell startup file (see Section 8.1.4). 


16.6Further Startup Topics 


Because this book deals only with the underlying Linux system, this chapter does not cover windowing 
environment startup files. This is a large issue indeed, because the display manager that logs youintoa 
modern Linux system has its own set of startup files, such as .xsession,.xinitrc, and the endless 


mess of GNOME-and KDE-related items. 


The windowing choices may seem bewildering, and this isn't far from the truth, because there isn't even 
one single common way to start a windowing environment in Unix. However, once you determine what 
your system does, you may get a little carried away with the files that relate to your graphical environment. 
This is fine, but don't carry it over to new users. The same tenet of keeping things simple in shell startup 
files works wonders for GUI startup files too. 


Chapter 17: Buying Hardware for Linux 


Overview 


Computers are frustrating. Simply shopping for one is enough to drive a sane person mad, and if you want 
a Linux machine, you might as well sign up for therapy now. Aside from relatively low hardware prices, the 
cards are stacked against you. A confusing barrage of specifications and incompatibilities awaits you in 
your quest, and along the way, you must often confront extremely lame advertising for what are often the 
ugliest examples of industrial design in history. 


This chapter is a field guide to hardware selection. Although hardware is in a constant state of flux, the 
good news is that an old adage applies: The more things change, the more they stay the same. If you can 
see computer hardware in more abstract terms, you have everything you need to make an educated 
purchase. Despite the constant escalation of specifications, a computer still has a processor (CPU), 
random-access memory (RAM), secondary storage, and some 


I/O devices. Every now and then, some underlying technology may change, but nothing is going to stop 
memory from doing anything other than storing a bunch of ones and zeros. 


Here are three guidelines that you should remember: 


Know what you want, and don't lose sight of it This usually comes down to a price versus 
performance trade-off, but details such as ergonomics can be just as important. 


Don't listen to vendors They generally want to sell you the most expensive thing in the store even 
though it may not even work correctly. In addition, these people are trained to sell systems for 
Windows users — in most cases they're not aware of hardware issues important to Linux users. 


You often get what you pay for However, refer to Rule 1 first: is the more expensive hardware what 
you really need and want? 


17.1Core Components 


The discussion in this chapter focuses on desktop computers. With the exception of notebooks, | build my 
machines from components. The goal is not to save money; in fact, the finished system often costs more 
than similarly outfitted pre-assembled computers. The advantage is that there are fewer headaches with 
custom configurations. If you Know exactly what goes into your computer, you will have no surprises, and 
in this territory, surprises only come in the unpleasant variety. You don't want hardware that Linux doesn't 
support, and an unreliable piece of hardware can seriously degrade your quality of life. 


Assembling a computer is easy, but it isn't for everyone. It takes time and a certain amount of dexterity to 
avoid hurting yourself on sharp electronic pieces. If you don't want to go for an off-the-shelf model that 
may contain unknown components, and you also don't want to assemble your computer yourself, you still 
have two choices. You can find out exactly what is in that off-the-shelf model that you're looking at, and 
then determine whether the parts will work with Linux. Reputable manufacturers list this information and do 
not substitute parts. Your other choice is to buy custom parts and then have a small computer shop or 
friend assemble them for you. If you go to a shop, make it clear that component substitution is 
unacceptable. 


You need the following components: 


= A motherboard. Modern motherboards have integrated hard drive interfaces, as well as USB and 
many other I/O ports. 


=» A processor (CPU). 

» Random-access memory (RAM). 
a» A hard disk. 

a An Ethernet NIC card (optional). 
=» A graphics card. 


=» A case to hold the components. The case usually includes a power supply. 


17.1.1Processor and Motherboard 


Virtually all 1A32-like (x86 PC) central processing unit (CPU) models work with Linux. The kernel and gcc 
compiler support many vendor-specific CPU optimizations, and features such as multiprocessing also work 
well with Linux. 


Two important considerations when choosing a processor should be power consumption and heat. CPUs 
that consume more power also generate a substantial amount of heat. To handle the extra heat, you need 
a lot of fans. Never overclock a processor running Linux, unless you like to see your computer crash. 


Note If you are interested in taming the usual din that a computer makes, a great place to start is the 
power supply. Most power supply units included with PC cases are of the very cheapest sort, with 
loud, unreliable cooling fans. You can easily replace the power supply with a low-noise premium 
model from a third party. It will not be cheap, but if noise irritates you, it is worth every penny. 


Although nearly all CPU models will work with Linux, some care must be taken when choosing the CPU's 
home — the motherboard. You should consider your motherboard, processor, and memory as a single 
unit, and you may even want to choose your motherboard first and find a CPU to fit. After all, it is possible 
that your motherboard will cost more than the CPU! 


Pick a motherboard that has been available for a reasonable amount of time. Three or four months should 
be sufficient. Often, initial releases of motherboards have bugs that cause mysterious hardware failures. 
Avoid fancy (untested) new features. Linux doesn't immediately support each new bell and whistle that 
comes off the assembly line, and these unusual features may interfere with normal system operation. 
Watch out for brand-new processor models that require you to purchase a new motherboard chipset, as 
these can be expensive nightmares. 


Modern motherboards come with a number of integrated peripherals. They normally include two ATA disk 
interfaces, plus parallel, serial, and USB ports. Some motherboards come with additional hardware, such 

as integrated SCSI controllers, video chips, and sound. These integrated features can save you space on 
the motherboard and the hassle of installing your own components, but if you buy separate plug-in 


components, you can perhaps save a little money and a compatibility headache. The advantages of 
integrated motherboard components are somewhat dubious in the case of most components except SCSI 
controllers. If, like many people, you choose a motherboard with integrated video, find out what chip it 
uses and make certain that the XFree86 server has support for it (See http://www. xfree86.org/ and 


http://www. xouv ert. org/). 


17.1.2Memory 


The third guideline from the beginning of this chapter ("you often get what you pay for") is most important 
when buying memory. Name-brand memory is only marginally more expensive than generic labels, and it 
can save you many headaches. A Linux machine exhibits odd behavior if it has bad memory. If a program 
seems to crash often, look for these symptoms: 


=» Thecrash happens in the same place every time. 

=» Rebooting makes the problem go away. 

=» The operating system appears unaffected, and other programs run fine. 
a It happens during frequent disk access. 


You should consider buying ECC (error-correcting code) memory. It is a little more expensive than 
standard memory, and not all motherboards support it (read your motherboard manual carefully), but ECC 
memory practically ensures that you never have a memory problem. 


A Linux system automatically uses extra memory for disk cache, and this can substantially improve 
performance for applications that use large files (or several small files). However, for many users, there is 
a point where more memory adds little or no benefit because the system has already cached nearly 
everything. You can briefly check the state of a system's memory utilization with the free command; the 
information in /proc/meminfo has a little more detail. 


17.1.3Hard Disk 


The old "IDE versus SCSI" question was once one of the bigger debates surrounding hard disks. However, 
if you want to build a desktop computer, the debate means little, because you will likely choose an ATA 
(IDE) disk out of cost concerns. In the old days, power users considered ATA disks to be performance 
dogs, and there wasn't a large disparity in price when compared against SCSI disks. Improvements in CPU 
speeds, caching, memory cost, the interface, and hard disk manufacturing techniques have made ATA 
perform tolerably well for a desktop system with a single disk — even Apple switched its desktop disks to 
the ATA type. 


A disk has two primary specification numbers: average access time (measured in milliseconds, or ms) and 
rotation speed (measured in rotations per minute, or RPM). In general, a faster (lower) access time is the 
more important consideration for Unix, because the system spends a lot of time looking at a bunch of 
small files. If you are a software developer, this is particularly important because compilers and scripting 
languages use many file accesses on a single invocation. On the other hand, if you have big files (movies, 
music, whatever), a higher rotational speed gets this stuff off the hard disk faster. 


There usually isn't a trade-off between rotational soeed and access time because they are linked. More 
RPMs mean that the disk head takes less time to pass over the proper place on the disk platter for the 
next access, hence a lower average access time. 


Higher performance, however, comes at a price above and beyond the sticker price. The fastest-spinning 
disks are usually noisy and generate more heat. I'll soare you the "hard disks | have hated in the past" war 
stories, but if you want a quiet disk, look for one with a fluid bearing system (Sometimes called FDB). 
These drives use liquid instead of solid ball bearings for their platters, and they are a big improvement for 
your ears. 


Any ATA disk should work in a Linux system. The kernel disk drivers are fast, and they support special 
features with the hdparm command. If you run into compatibility issues, they will likely be between your 
motherboard and the disk (in particular, some older motherboards do not support disks larger than 32GB; 
look for jumpers on the disk to "clip" the disk, limiting it to a lower capacity). 


If you put two ATA disks in the same system, put them on different interfaces (your motherboard should 
have primary and secondary ATA interfaces), which should improve concurrent access slightly. If you 
purchase an add-in ATA interface card, you may need extra drivers. When considering one of these cards, 
look for drivers in the ATA support menu described in Section 10.4.1. 


17.1.4Network Cards and Infrastructure 


The Linux kernel supports most wired PC Ethernet devices (also known as NICs, or network interface 
cards), but you should check any card against the compatibility list in a current Linux kernel. Don't believe 
a manufacturer's claim of compatibility in this case — the manufacturer may require you to install a special 
kernel driver before the device works. Keep in mind that the kernel drivers that come from vendors often 
don't work correctly. 


Most cable and DSL adapters require an Ethernet card. You should be able to use any supported Ethernet 
card regardless of the protocol that you use, provided that Linux supports the protocol. You may need to 
use a funny protocol like PPPoE for your provider (See Section 5.9). 


Current networks of 100Base-T (100 megabits per second, or about 12 megabytes per second) are more 
than adequate for most purposes. They are also very inexpensive. To attach several machines to a home 
network, you need a hub or switch (it is possible to connect two machines directly with an Ethernet 
crossover cable, but this isn't really worth the trouble). 


The most significant differences between using a hub and a low-end switch is that a switch can send and 
receive signals at the same time and therefore has a potential of faster throughput and lower latency. 
However, unless you have a habit of sending large files around your network simultaneously, you won't 
notice much of a performance difference between the two. On the other hand, the difference in price 
between hubs and switches is also fairly small. Whether you purchase a hub or a switch, pick a model that 
has ports for each of your computers, plus a few extra. Also, make sure that the hardware supports the 
older 10Base-T standard. This feature is often called 10/100, and it provides the most convenient way to 
move files from an old computer with a 10Base-T interface. 


Wireless Ethernet 


You may want to consider wireless Ethernet for a notebook, even if you don't plan to use Ethernet when 
you're on the go. There are several advantages: 


a It is one less cable to connect. This saves time as well as wear and tear on the machine. 
a Visitors with wireless cards can connect to your network without much hassle. 
» Some PDAs can connect to a network with wireless Ethernet. 
» You may need to connect to a foreign network at some time. 
There are also some disadvantages: 
a It is more expensive. 
= Security can be a hassle (see Section 5.15.1). 


=» There are some really annoying twerps who think that wireless Ethernet makes them the coolest 
people on the planet. 


To make wireless Ethernet work, you need a wireless access point along with the network card (some 
notebooks come with integrated cards). Many access points come in units that also provide a broadband 
router and an integrated switch. Even if you don't have a broadband connection or choose not to use the 
routing feature, the switch can still be useful for connecting desktop computers. 


For notebook Ethernet cards, see Section 11.6. For information on how to set up a wireless client, see 
Section 5.15. 


17.1.5Graphics Hardware 


Your primary concern in choosing a video card should be compatibility with XFree86. Although you can 
inspect the list of compatible cards at http://www. xfree86.org/ and http://www. xouvert.org/, it is 
unfortunately not always easy to identify a card because manufacturers have a habit of not changing 
product names while altering hardware significantly. 


As with the motherboard, a good method for ensuring that your graphics hardware is compatible with your 
system is to identify a brand or particular graphics chip that has consistent support and then find a video 
card that isn't cutting-edge. If a card has been available for some time, there is a much greater chance of 
XFree86 support. Regarding performance, quick video hardware often means very little under Linux; in 


fact, the fastest 3-D hardware may not be optimal for normal tasks in XFree86. 


XFree86 supports multi-output video cards and additional cards that can drive multiple monitors, 
sometimes called multiple heads. If you value your onscreen workspace, this is worth looking at because 
the emergence of LCD displays makes it feasible to have two large monitors without a desk the size of an 
aircraft carrier. You can also combine multiple displays into one big virtual display with the Xinerama 
extension. 


17.2Other Hardware Components 


Now it's time to move beyond the essential components and look at your choices for the other parts that 
you can attach to a computer. 


17.2.1Monitors 


Because you will work with the shell interface more under Unix than other PC operating systems, you 
should choose a monitor that displays crisp text. In addition, high resolution is important in a monitor 
because you want the ability to display at least two shell or text editor windows at once, so that you can 
quickly switch between them. However, you can only take resolution so far. To the beginner, itis always 
tempting to reduce the font size to accommodate more windows, but because this increases eye strain, 
there is no real substitute for a large display. 


Traditional Unix workstations like Sun SPARCstations usually came with very large, heavy, expensive 
cathode ray tube (CRT) monitors offering some of the highest resolutions commercially available. CRT 
displays are now lighter and carry the lowest price tags of high-end displays. However, you should avoid 
them whenever possible. Above all, an LCD display offers a razor-sharp text display that is impossible on 
any CRT. This advantage, combined with a very light weight, small footprint, and lack of a flyback 
transformer makes a good case for an LCD display for all but the most special-purpose applications. (At 
some point in the not-so-distant future, a CRT will not even be a consideration, rendering this paragraph a 
historic curiosity.) 


As mentioned above, traditional monitors that came paired with Unix machines had fairly high resolutions. 
The minimum that Sun and NeXT used was around 1152 x 900 (often called megapixel displays ), and 
most users found it comfortable (you won't find this option on modern LCD displays). 1024 x 768 is a little 
cramped but passable if you can keep a minimal desktop. It also may be the only option if you are 
shopping for a notebook. A 1280 x 1024 display is the next step up and it is often ideal; if you happen to 
have just a little extra money and are itching to spend it on something worthwhile, this is it. Higher 
resolutions such as 1600 x 1200 are good if you have large graphics to display. However, if you want to 
increase your workspace, especially for text, it's better to buy a second monitor and use the multi-monitor 
support mentioned in Section 17.1.5. A multi-monitor configuration costs less than a single huge monitor 
and gives you more effective onscreen space due to the layout of the monitors. 


17.2.2Keyboards 


Choose your keyboard wisely, because you'll use it more than you ever imagined in Linux. First, check the 
basics of the layout. Is the ESC key in a reasonable place, and is it a normal-sized key? This is extremely 
important for vi users, who press ESC on a regular basis. Where are the tilde-backtick (~~) and pipe- 
backslash (| \) keys? These characters come up frequently in shell commands. (To those using non-U.S. 
keyboards, you might want to ask a friend about good key layouts and mappings.) 


If you intend to run only Unix on your computer, keys other than those immediately around the 
alohanumerics aren't very important. You don't really need a numeric keypad or 37 function keys. Arrow 
keys are also slightly less important when running Unix unless you're playing a game. Nearly every 
program that takes text input accepts CONT ROL-P, CONTROL-N, CONTROL-B, and CONTROL-F (up, 
down, left, and right, respectively). In vi, you will make use of h, j, k, and | (left, down, up, and right). 


One little thing to watch out for is the CAPS LOCK key. On virtually every keyboard made today, it is next 
to the A key. This is wrong; the CONTROL key belongs there. Philosophical issues aside, see if the CAPS 
LOCK key is large enough and doesn't have an overly weird shape. You should remap the CAPS LOCK 
key to CONTROL and use it that way. 


17.2.3Mice 


Other than your personal preferences and testing, there is only one important thing to remember when 
buying a mouse: Make sure your mousehas at least three buttons. The third (middle) button in X Window 
System applications does a paste operation; you don't want to be without it. You can even use it to paste a 
URL into a browser window. Also, clicking on a link in Mozilla with the middle button makes the link appear 
in a new window or tab. 


In a modern mouse, the scroll wheel usually doubles as the third button when pressed down. The scroll 
wheel is more difficult to press than a real button, and many Unix veterans dislike this. However, there 


may be some merit to it, because clumsy fingers on traditional three-button mice have been known to 
paste strange commands into Unix shells and editors. 


17.2.4Modems 


Most fax modems and many voice modems work with Linux. There is only one very important rule when 
buying a modem for Linux: Never buy aWinmodem. These devices require complicated software drivers 
to do signal processing. There are some straggling Linux drivers for certain Winmodems out there, but you 
shouldn't bother. You don't want to waste time trying to set this up, and you don't want the extra strain on 
your kernel. It is unfortunate that many notebooks come with Winmodems; if you have a Winmodem in 
your notebook, your only option is usually to get a PC Card modem. 


That said, for desktops you have a choice between internal and external modems. Internal modems are 
cheaper, do not have external power supplies, and need no cords other than the telephone line. 
Unfortunately, they may also require some Plug-and-Play manipulation or disabling of built-in serial ports. 
You may be able to use an external USB modem, but make sure that there's a Linux driver before you 
buy. 


17.2.5Printers 


Because Unix printer drivers generate the PostScript page description language, the easiest way to get a 
Linux-compatible printer is to buy one that supports PostScript. Many laser printers and some inkjet 
printers come with this capability. Unfortunately, these printers are also more expensive than those with 
simpler imaging technology. 


Linux also supports printers that don't understand PostScript by using Ghostscript to transform the 
PostScript generated by the printer drivers into something that these printers understand. However, this 
adds an extra step to the printing process inside your machine, and it can be difficult to configure (see 


Chapter 12). 


There are three common ways to connect a printer to a Linux system: using a parallel port, using a USB 
port, or adding the printer to a local area network. The parallel port is an old and somewhat primitive 
method, but it works well. USB is the newest method and should work as well as your USB drivers do (see 
Section 11.3). Printing over the network is fast, it requires no extra kernel drivers, and it is often the 
easiest to configure. Most network-capable printers also support PostScript, but they are considerably 
more expensive than their non-networked counterparts. 


17.3A Word About Hardware Upgrades 


Before we look at how you can save money, you should know something about upgrades. For many 
people, some kinds of products seem more appealing because you can upgrade them somewhere down 
the line. This may work for old tools and kitchen appliances, but the computer industry doesn't think that 
way. They want you to buy something, use it for a year, throw it away, and then buy the newest model 
because they wouldn't make any money if they actually had to offer upgrades for the dinosaurs of the past. 
Really, computers just aren't designed like the Stanley #5 jack plane, which was offered in the same basic 
form for 115 years. 


If you see something advertised as being upgradable to some future specification with some extra piece of 
hardware, it probably isn't going to do you a whiff of good, especially if that extra piece of hardware doesn't 
exist yet. It usually means that the company has a newer product that isn't quite ready, and therefore, the 
marketing department is trying to think of ways to milk an old product. If that little upgrade actually does 
appear, it will most likely be crippled because the old and new technologies are so different that the 
engineers had to do some really bogus stuff to get the things to talk to each other. | could bring up 
countless examples, many involving CPUs, but you should take my word on this one. 


So what can you upgrade, and what kinds of parts can you carry from an old computer to a new one? 
Disks, for one, are always fairly easy to add to a machine, whether a disk is new or used. Memory is a little 
more complicated. For the most part, you can always add more memory to your existing system, but keep 
in mind that you may need to remove some of your old memory to make room for the new stuff. 


You should regard the motherboard, processor, and memory as one unit when purchasing hardware. Don't 
even think of carrying just one of these components over to a newer system (well, this is a slight fib; 
sometimes you can keep your memory, but it's very rare). You may also need a new power supply, case, 
or graphics card when you upgrade other hardware on your system, because the standards for these 
components are not terribly stable. 


Peripherals and USB devices are easier to carry from system to system. There really isn't any need to 
change peripheral devices (like printers) if what you have works. And if you have a keyboard or mouse 
that you like a lot on your old system, then in the name of your own sanity, don't get rid of it. | hate to say 
this, but they just don't build keyboards like they used to. It might be worth buying a special adapter to 
make an old keyboard compatible with a new system (for example, to convert an AT keyboard to USB). 


Upgradable firmware (embedded software) in external hardware components (Such as modems) is also a 
good thing, because it's much easier for a company to write software to support new protocols than to 
develop and produce hardware fixes. 


17.4Saving Money 


Axiom 3 in the introduction to this chapter stated that you often get what you pay for; in other words, you 
might need to pay for a system that works and works well. However, there's no injunction against saving 
money. If you don't have an obsession with the very latest products, and you know what to look for, you 
can look for slightly older stuff and save a substantial sum. 


17.4.1CPU 


Although it is true that your CPU does most of the work on a Linux system, the CPU normally spends most 
of its time doing absolutely nothing. The uptime command usually reveals that the processor isn't running 
any jobs. If processor speed is important to you, you'll know it from the type of computing that you do, be it 
graphics or numerical analysis. But for the vast majority of users, there are many features more important 
than CPU speed, and carefully choosing a processor is one of the best ways to save money on a new 
computer. 


Nowhere does price so wildly diverge as among processors. A vendor's list at the time of this writing shows 
one price for its fastest CPU. The next step down is 50 percent of that price, but it is still 91.5 percent as 
fast as the top model. A slower processor costing 25 percent of the top model retains 78 percent of the 
performance. Now you may ask yourself, maybe | need that extra 22 percent of power? For comparison, | 
am typing this on a computer that is only 12 percent as fast as the top processor in that list, and | have no 
complaints. 


The trick to selecting a processor is to pick a spot on the performance/price curve that looks good. Figures 
17-1 and 17-2 show plots of ten processors. Figure 17-1 shows how quickly the price rises as the processor 
speed reaches the fastest available. In Figure 17-2 on the next page, you can see how much performance 
you get for each dollar that you spend for the same set of processors. 
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Figure 17-1: Costs of several processors. 
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Figure 17-2: Performance value. 


If you are strictly interested in getting the most bang for the buck, the best choice is the CPU represented 
by the maximum in the graph on the right. However, keep in mind that you may want to go for one of the 
CPUs with slightly faster numbers because it may cost only a few dollars more, or you might be able to 
buy a CD or something else with the savings from choosing a marginally slower option. 


You must be cautious in measuring the performance of a processor. The only halfway precise way to 
compare processors is to compare different clock speeds of the same model. Looking at the megahertz 
(MHz) numbers of two different models, even from the same manufacturer, is comparing apples and 
oranges. The benchmark results you read about are also unreliable because operation speeds vary among 
different processors. Unless you know exactly what you're looking for, benchmarks are next to useless. 
Decide on a price first, and then look for the best deals among the models that fall within your price range. 


If you find any of this economic analysis remotely interesting, you might want to read the first chapter of 
Computer Architecture: A Quantitative Approach [Hennessey]. 


17.4.2Hard Disk 


More expensive hard disks often provide more space for the money. For example, a certain hard disk may 
offer 100 percent more space over another model for only 33 percent more money. But if you're running 
Linux, which is an operating system with relatively low system storage requirements, is "Supersizing" really 
worth it? Considering how much a modern hard disk can store, will you actually use that extra space 
before you buy another disk? 


By the time you need to purchase another disk, you may find that the cost per megabyte on a new disk is 
far lower, or that there are desirable technology improvements that may just make you want to 
decommission your old disk. Remember that the hard disk still has moving parts, and it is therefore one of 
the least reliable components in a computer. 


17.4.3Computer Case 


Ugly cases with unrefined power supplies and unsophisticated internal structures are cheaper. If you're not 
picky and can put up with the hassle, this could be worth it (that is, you're a graduate student and know 
what it means to have something "cheap but somewhat functional if not kind of annoying"). Also, you 
might be able to get a used case for free. Be careful, though: a secondhand power supply may not work 
with your motherboard, and it may not work at all. 


17.4.4Video Card 


Sometimes you can get a graphics card that the vendor targets for office users. These cards are often less 
expensive than others. Also, old or discontinued models on clearance won't hurt performance on Linux. 


17.5Notebooks 


It is difficult to balance Linux compatibility with the latest features on notebook computers. The most 
common problems with notebooks are built-in video chipsets that XFree86 does not support. Other 
components that tend to lack drivers are modems, Ethernet interfaces, sound cards, and CD-ROM drives. 


The online community is a good place to look for information on Linux notebook compatibility. The 
http://www.tuxmobile.org/ site is a clearing-house of links to people who are running Linux on nearly all 
common notebook models. However, these pages are often out of date, and the driver configuration or 
bug fixes described may not be necessary with current Linux drivers. 


17.6Smaller Designs 


Once upon atime, if you wanted a PC, your options were limited to large machines. Because 
interchangeable parts were so important to build the PC as a commodity, manufacturers were reluctant to 
design anything smaller because the form factor would change and be incompatible with everything on the 
market. Very small Unix machines were on the market, such as the Sun SPARCstation IPC and IPX, but 
they were expensive. 


This is changing. Increased demand from consumers, small dimensional standards taken largely from 
notebook computers, and special applications have created a market for a newclass of machines. Form 
factors such as Mini-ITX are now on the market. They aren't the fastest computers around, and they aren't 
overly expandable, but they have big advantages. They are small, quiet, use less power, generate less 
heat, and are astonishingly inexpensive. Linux has made a contribution to the increasing popularity of 
these machines. Whereas a computer with relatively low processor power stood no chance against a 
market using Microsoft's resource-hungry operating systems, the ever-increasing population of Linux users 
does not care. Manufacturers know this and make sure that their products support Linux. 


The current small machines are hybrids. The motherboards can still fit into standard cases, use normal- 
size memory, and often have a single PCI slot. They accept standard PC components, even the power 
supply. However, because many of the components are self-contained and generate little heat (including 
power supplies), it has become fashionable to custom-build smaller enclosures. These machines really 
shine when placed in special small cases that will not accept larger components. 


One common example of this is the CD-ROM drive. On a regular desktop computer, these devices are 
gigantic because they fit into a half-height 5 1/4-inch drive bay. To make them fit in a smaller machine, an 
easy substitution for the normal size CD-ROM drive is a notebook CD-ROM drive. Like the embedded 
processors found on many small motherboards, a notebook CD-ROM drive may not be as quick as its 
larger cousin, but it is a fraction of the size. 


Chapter 18: Further Directions 


There's alot to Linux. This book can only go so far, and although it's time to wind down, this chapter will 
give you an idea of what lies beyond the fundamental topics covered in this book. 


18.1Additional Topics 


Most (but not all) of these topics are network related. Furthermore, none are specific to Linux; you can 
carry the knowledge you have of them to and from other Unix systems. 


Electronic mail Email is a big topic. Chapters 5 and 6 talked a little about the SMTP port (25) used 
for mail transfer, and they briefly mentioned Postfix and qmail, the two most viable mail transfer 
agents (MTAs) available now. There's more behind the MTA: for example, if you want to run an IMAP 
(Internet Message Access Protocol) server, you probably want the Cyrus package. 


Domain Name Service (DNS) Setting up a client is easy, as you saw in Chapter 5, but setting up a 
DNS server is trickier. The most common DNS server is BIND (Berkeley Internet Name Domain), but 
a newer DNS server named djbdns is also gaining popularity. 


Web servers In the Unix world, "Web server" usually means Apache, the most popular Web server in 
the world. However, many scripting languages and other tools have embedded Web server features 
that can aid debugging, status monitoring, and other tasks not directly related to serving Web pages to 
the general public. 


Virtual private networks (VPNs) A VPN allows you to connect several widely dispersed machines in 
a virtual network (for instance, a single subnet). Administrators typically create a VPN in conjunction 
with the IPSec protocol so that they can disperse sensitive data (that might otherwise be behind a 
network firewall) through an encrypted network channel across a possibly insecure link. 


Thescreen program Did you ever want to take your shell session wherever you go? You can do it 
with the screen program, a terminal multiplexer that supports multiple shells, disconnection, 
reconnection, scrollback, cut and paste, and more. This utility is a perennial favorite among seasoned 
Unix wizards. If you want some of the same functionality with a windowing system, check out VNC 
(Virtual Network Computing). 


DB files and DBM files When using many larger servers, you will encounter binary DB and DBM 
files. These files are very small databases that store blocks of data keyed by small strings, usually as 
some kind of hash table. For whatever reason, there are an absurd number of different DB/DBM file 
formats (DB, GDBM, NDBM, and SDBM, to name a few). Most packages that use DBM-style files 
come with their own manipulation utilities (for example, the postalias command that comes with 
Postfix), but scripting languages such as Perl include powerful interfaces that you can use if you need 
to do something beyond the ordinary with these database files. 


Relational databases You can run the powerful MySQL and PostgreSQL database servers on Linux. 
There are several commercial servers as well, but this book intentionally avoids talk of commercial 
software products. 


RCS and CVS The Revision Control System (RCS) is a way to track file revisions. Originally meant 
for software development, many systems administrators use RCS to control important system files so 
that they have a record of who made each change (and in theory, so that they can back out of harmful 
changes, though this rarely comes up in practice). The Concurrent Versions System (CVS) isa 
network-extensible revision control system that can handle several versions of the same package at 
once. 


Pluggable Authentication Modules (PAM) If you don't like the system of passwords in 
/etc/passwd, you Can replace it with a PAM module that supports some other scheme. 


Network Information Service (NIS) NIS is a terrifying combination of RPC and DBM files, used by 
many systems to provide networked information such as passwd and group. It is an old system that 
isn't particularly robust, secure, or pleasant, but it is supported by most Unix platforms. 


Kerberos This is a powerful network authentication system. 


Lightweight Directory Access Protocol (LDAP) LDAP serves a bunch of data entries to network 
clients (as NIS does), except that where NIS allows only a single key per data entry, LDAP supports a 
hierarchy of keys for entries. LDAP is very extensible; OpenLDAP is a popular open source LDAP 
server. 


Network File System (NFS) This is the traditional method of sharing files across a network on a Unix 
cluster. NFS is a somewhat clunky system that is still in use because most Unix systems support it. 
Plus, the alternatives such as AFS (Andrew File System), DFS (Distributed File System), and 9P are 
too expensive, difficult to configure, and not terribly well understood. 


Secure Sockets Layer (SSL) Many network servers now offer SSL support to authenticate and 
encrypt individual TCP connections with the same technology used for secure Web sessions. If you 
want to set up a server that supports SSL, be prepared to learn all about public keys, private keys, 
certificates, signing, and more. 


18.2Final Thoughts 


One of the difficulties of Linux is that, like most software, certain technical details change all of the time 
because software maintainers change, they implement new features, and they can be just plain fickle. 
Although this book strives to cover topics that have some degree of stability, it's impossible to tell the 
future. The good news is that the base system really doesn't change much. Concentrate on what the 
system components do, not the details of every strange little option. Remember that the online 
documentation is always there. In many cases, learning the correct terminology is more important than 
learning the actual details, because it's the terminology that helps you in your search for the 
documentation. 


Linux (and Unix in general) doesn't exist in a vacuum. There are a lot of people out there who can help 
you. A few examples are IRC channels, mailing lists, user groups (check http://www. linux.org/groups/), and 
the USENIX/SAGE organizations (http://www. usenix.org/). USENIX has been around since the early days 
of Unix (incidentally, if you're interested in "the old days," you should read AQuarter Century of UNIX 
[Salus]). 


Keeping your system simple and clean helps when tracking problems. Don't get obsessed with it, though 
— zealously removing "errant" files can be harmful. 


Finally, don't be afraid to try new stuff! 


Appendix A: Command Classification 
This appendix is a guide to the common standard commands. They are organized in the following tables: 


Table A-1: File Management Commands 




















Command | Description 

chgre | Changes a file's group — 0000 
chmod Changesafile'spermissions 000 
chown | Changes a file's user ownership 

op Copies afie O00000 
ad Converts and copies O O O 0000 
af Displays disk usage statistics 0000 
du | Displays directory space usage 

file Identifies a filetype OOOO 
iiia Searches for afile 00 
In Creates a symbolic or hardlink = sss 
ls | Lists files 

mkdir Creates a directory —ss—‘“—s—s—s—s—sSSSSCSCSCiCS 
mkfifo | Creates anamed pipe — 0000 
mknod | Creates a special file 0000 
mv | Renames or moves a file 

rm Removes a file 0000 
toucn [Creates afile orupdates a file'stimestamp č  ěăě  ăž  ă o ăě = < ć | 





Table A-2: Text Processing and Scripting Commands 




















Command | Description = 
cat | Displays and concatenates files 

cmp Comparesbinayfiles 00000 
cut Extractscolumnsoflines sss 
Cutt | Compares text files 

dirname | Extracts the directory from a filename 

echo Prntstext 
egrep | Extended grep 

false | Returns a nonzero exit code ——ss—‘“—sSSSSCSCis™S 
fmt | Breaks long lines and reformats text 

grep ‘Searches for lines matching a regular expression = 























grof£ | A multi-purpose typesetting utility = ss 
head Displays the first lines of afie = sss—<“—sSsSSSSOSSCisS 
ispell | A spelling checker 

less | Displays a text file 

more lDisplaysatextfle sis 
nroff | Formats roff documents for text display ss ss—SsSCS 
parch | Incorporates changes into files; the opposite of diff 

sed A stream editor O 00 
sort Sorts lines in afie 000 
split | Chops a file into pieces 

ai “Displays the last lines of a file 00000000 
bee Duplicates a file stream —ssss—<“—sSsSsSSSC‘i‘CS 
test ([) Checks a condition 

cr | Translates (or substitutes) characters 

true Returns an exit code of 0 (true) 

wT Removes duplicate adjacent lines sss 
r “A visual full-screen editor ssi 
we | Counts words, lines, and characters in a file 





Table A-3: Online Documentation Commands 


info 


man 





Command [Description = 


Displays GNU-style documentation 
Displays the traditional Unix online manual 





Table A-4: Process and System Utility Commands 





at 


chin 


Command | Description 


Runs a program at a certain time 
Changesfinger information 





chsh 


groups 


id 


crontab 


| Changes shells 


Runs a periodic job 
Shows group membership 
Shows the current user ID 





kill 
logger 
login 


lsof 


| Sends a signal to a process 


Records a message to the system logger 


Allows a user to login 


Lists open files and other information 





mount 
newgrp 


nice 





| Attaches a filesystem to a directory tree 


Changes the current default group 
Runs a process with a suggested priority 


passwd Changes a password 
printenv Prints environment variables 


ps | Displays processes 



















renice | Changes the suggested priority for a process 

reset | Attempts to reset the terminal 0000002 
strace [Traces system cais O00000 
su | Switches users 


sync Writes kernel buffers to disk 
time Displays how much processor and system time a process takes 
top Shows the processes with the most resource consumption 


umount | Detaches a filesystem from a directory tree 











Table A-5: System Information Commands 


Command ‘Description = 


arch | Displays the system architecture 








df | Displays disk usage Statistics 


dmesg Ppicsawbiiccdkemcdtessaacs| aaee) 
finger | Displays user information = ssss—<“—sSsSSSCSCSCisS 
free | Displays free memory statistics 

hostname [Displays the current host's name OOO 
last iowstieiemeswioloadcain O eee E] 
tty Displays the current terminal name sss 
uptime | Displays system load and how long the system has been running 

vmstat | Displays virtual memory statisties sss 











who | Displays current users 


whoami Displays the current user 








Table A-6: Archival and Compression Commands 





Command | Description 


bunzip2 A decompression program 
bzip2 A decompression program 


epic | An archival program 













gunzip A decompression program 
gzip A compression program 


tar An archival program 


uncompress | A decompression program 


unshar A de-archival program 











uudecode A decoding program (the counterpart of uuencode) 
uuencode Encodes binary file into a text file 
zcat | Decompresses into a file stream 





Table A-7: Miscellaneous Utility Commands 


Command ‘Description = 
be | A simple calculator 

cal Shows a calendar  sss—“—sSsSSSOSCSCSCSCSCSCi‘S 

date Displays the currentdate — 0000 

dc Runs the RPN calculator —sss—<“—sSSSSSCSCSCidS 
pwd | Prints the working directory 

Shes ap Shu [Starts a shell where all outputisrecordedinafile ë ë ë O ă O ăë ë <ë = ë | 

sleep [Pauses fora specified numberof seconds —  ěăě  ž  ăž  žăž ë <ë | 


strings | Attempts to show any text embedded in a binary file 














yes | Prints an endless stream of lines 


which Displays the first matching program in the current path 








Table A-8: Development Commands 





Command | Description 


ar A library archiver 
m An assembler  —s—s—sSSsSC‘“‘“‘“‘sésSCSCsS 
C++ A C++ compiler 


Ce AC compiler 











cpp A C preprocessor 

g++ A C++ compiler (see c++) 
gec | A C compiler (see cc) 
gdb | The GNU debugger 


install | Copies a file into a location with certainparametes — O OOOO 
ld iker 
ldd | Displays dynamic libraries 

hake A package-building tool —sss—s—sSSSSCisS 











Table A-9: Shells 


Command Description 

bash | The Bourne Again Shell 
csh The C Shell 
ksh The Korn Shell 


sh | The Bourne Shell 
| The TC Shell 
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